From a1fbafa63f97ec066a4fffdd407d9810b589b190 Mon Sep 17 00:00:00 2001 From: Daya Adianto <dayaadianto@cs.ui.ac.id> Date: Tue, 3 Sep 2019 13:21:43 +0700 Subject: [PATCH 01/53] Remove debug log files --- debug.log | 6 ------ npm-debug.log.1735884061 | 0 2 files changed, 6 deletions(-) delete mode 100644 debug.log delete mode 100644 npm-debug.log.1735884061 diff --git a/debug.log b/debug.log deleted file mode 100644 index 622e4f65..00000000 --- a/debug.log +++ /dev/null @@ -1,6 +0,0 @@ -[0419/113046.282:ERROR:process_info.cc(625)] range at 0x953e621000000000, size 0x202 fully unreadable -[0419/113046.284:ERROR:process_info.cc(625)] range at 0x953e623000000000, size 0x202 fully unreadable -[0419/113046.284:ERROR:process_info.cc(625)] range at 0x0, size 0x202 fully unreadable -[0419/113046.299:ERROR:file_io_win.cc(145)] CreateFile C:\Users\baisa\AppData\Local\Temp\karma-28613191\Crashpad\settings.dat: The system cannot find the path specified. (0x3) -[0419/113046.299:ERROR:file_io_win.cc(154)] CreateFile C:\Users\baisa\AppData\Local\Temp\karma-28613191\Crashpad\reports\a859f38f-0a5a-4ff9-939d-aac920dec0cd.dmp: The system cannot find the path specified. (0x3) -[0419/113046.299:ERROR:crash_report_exception_handler.cc(95)] PrepareNewCrashReport failed diff --git a/npm-debug.log.1735884061 b/npm-debug.log.1735884061 deleted file mode 100644 index e69de29b..00000000 -- GitLab From 3bce819c8ef7de6502d22f944aadd7e91ff5207d Mon Sep 17 00:00:00 2001 From: Daya Adianto <dayaadianto@cs.ui.ac.id> Date: Tue, 3 Sep 2019 14:09:35 +0700 Subject: [PATCH 02/53] Update .gitignore files --- .gitignore | 336 ++++++++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 321 insertions(+), 15 deletions(-) diff --git a/.gitignore b/.gitignore index e52ab627..2c6cae64 100755 --- a/.gitignore +++ b/.gitignore @@ -1,15 +1,321 @@ -node_modules -*.pyc -.pyc -venv -assets/bundles/* -.idea -webpack-stats.json -.npm -.bash_history -.cache -test/* -.coverage -.tmp/ -npm-debug.log -debug.log +# Created by https://www.gitignore.io/api/node,python,pycharm+all,visualstudiocode +# Edit at https://www.gitignore.io/?templates=node,python,pycharm+all,visualstudiocode + +### Node ### +# Logs +logs +*.log +npm-debug.log* +yarn-debug.log* +yarn-error.log* +lerna-debug.log* + +# Diagnostic reports (https://nodejs.org/api/report.html) +report.[0-9]*.[0-9]*.[0-9]*.[0-9]*.json + +# Runtime data +pids +*.pid +*.seed +*.pid.lock + +# Directory for instrumented libs generated by jscoverage/JSCover +lib-cov + +# Coverage directory used by tools like istanbul +coverage +*.lcov + +# nyc test coverage +.nyc_output + +# Grunt intermediate storage (https://gruntjs.com/creating-plugins#storing-task-files) +.grunt + +# Bower dependency directory (https://bower.io/) +bower_components + +# node-waf configuration +.lock-wscript + +# Compiled binary addons (https://nodejs.org/api/addons.html) +build/Release + +# Dependency directories +node_modules/ +jspm_packages/ + +# TypeScript v1 declaration files +typings/ + +# TypeScript cache +*.tsbuildinfo + +# Optional npm cache directory +.npm + +# Optional eslint cache +.eslintcache + +# Optional REPL history +.node_repl_history + +# Output of 'npm pack' +*.tgz + +# Yarn Integrity file +.yarn-integrity + +# dotenv environment variables file +.env +.env.test + +# parcel-bundler cache (https://parceljs.org/) +.cache + +# next.js build output +.next + +# nuxt.js build output +.nuxt + +# vuepress build output +.vuepress/dist + +# Serverless directories +.serverless/ + +# FuseBox cache +.fusebox/ + +# DynamoDB Local files +.dynamodb/ + +### PyCharm+all ### +# Covers JetBrains IDEs: IntelliJ, RubyMine, PhpStorm, AppCode, PyCharm, CLion, Android Studio and WebStorm +# Reference: https://intellij-support.jetbrains.com/hc/en-us/articles/206544839 + +# User-specific stuff +.idea/**/workspace.xml +.idea/**/tasks.xml +.idea/**/usage.statistics.xml +.idea/**/dictionaries +.idea/**/shelf + +# Generated files +.idea/**/contentModel.xml + +# Sensitive or high-churn files +.idea/**/dataSources/ +.idea/**/dataSources.ids +.idea/**/dataSources.local.xml +.idea/**/sqlDataSources.xml +.idea/**/dynamic.xml +.idea/**/uiDesigner.xml +.idea/**/dbnavigator.xml + +# Gradle +.idea/**/gradle.xml +.idea/**/libraries + +# Gradle and Maven with auto-import +# When using Gradle or Maven with auto-import, you should exclude module files, +# since they will be recreated, and may cause churn. Uncomment if using +# auto-import. +# .idea/modules.xml +# .idea/*.iml +# .idea/modules +# *.iml +# *.ipr + +# CMake +cmake-build-*/ + +# Mongo Explorer plugin +.idea/**/mongoSettings.xml + +# File-based project format +*.iws + +# IntelliJ +out/ + +# mpeltonen/sbt-idea plugin +.idea_modules/ + +# JIRA plugin +atlassian-ide-plugin.xml + +# Cursive Clojure plugin +.idea/replstate.xml + +# Crashlytics plugin (for Android Studio and IntelliJ) +com_crashlytics_export_strings.xml +crashlytics.properties +crashlytics-build.properties +fabric.properties + +# Editor-based Rest Client +.idea/httpRequests + +# Android studio 3.1+ serialized cache file +.idea/caches/build_file_checksums.ser + +### PyCharm+all Patch ### +# Ignores the whole .idea folder and all .iml files +# See https://github.com/joeblau/gitignore.io/issues/186 and https://github.com/joeblau/gitignore.io/issues/360 + +.idea/ + +# Reason: https://github.com/joeblau/gitignore.io/issues/186#issuecomment-249601023 + +*.iml +modules.xml +.idea/misc.xml +*.ipr + +# Sonarlint plugin +.idea/sonarlint + +### Python ### +# Byte-compiled / optimized / DLL files +__pycache__/ +*.py[cod] +*$py.class + +# C extensions +*.so + +# Distribution / packaging +.Python +build/ +develop-eggs/ +dist/ +downloads/ +eggs/ +.eggs/ +lib/ +lib64/ +parts/ +sdist/ +var/ +wheels/ +pip-wheel-metadata/ +share/python-wheels/ +*.egg-info/ +.installed.cfg +*.egg +MANIFEST + +# PyInstaller +# Usually these files are written by a python script from a template +# before PyInstaller builds the exe, so as to inject date/other infos into it. +*.manifest +*.spec + +# Installer logs +pip-log.txt +pip-delete-this-directory.txt + +# Unit test / coverage reports +htmlcov/ +.tox/ +.nox/ +.coverage +.coverage.* +nosetests.xml +coverage.xml +*.cover +.hypothesis/ +.pytest_cache/ + +# Translations +*.mo +*.pot + +# Django stuff: +local_settings.py +db.sqlite3 +db.sqlite3-journal + +# Flask stuff: +instance/ +.webassets-cache + +# Scrapy stuff: +.scrapy + +# Sphinx documentation +docs/_build/ + +# PyBuilder +target/ + +# Jupyter Notebook +.ipynb_checkpoints + +# IPython +profile_default/ +ipython_config.py + +# pyenv +.python-version + +# pipenv +# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control. +# However, in case of collaboration, if having platform-specific dependencies or dependencies +# having no cross-platform support, pipenv may install dependencies that don't work, or not +# install all needed dependencies. +#Pipfile.lock + +# celery beat schedule file +celerybeat-schedule + +# SageMath parsed files +*.sage.py + +# Environments +.venv +env/ +venv/ +ENV/ +env.bak/ +venv.bak/ + +# Spyder project settings +.spyderproject +.spyproject + +# Rope project settings +.ropeproject + +# mkdocs documentation +/site + +# mypy +.mypy_cache/ +.dmypy.json +dmypy.json + +# Pyre type checker +.pyre/ + +### VisualStudioCode ### +.vscode/* +!.vscode/settings.json +!.vscode/tasks.json +!.vscode/launch.json +!.vscode/extensions.json + +### VisualStudioCode Patch ### +# Ignore all local history of files +.history + +# End of https://www.gitignore.io/api/node,python,pycharm+all,visualstudiocode + +# Project-specific ignored files +.bash_history +webpack-stats.json +assets/bundles/* +test/* +.tmp/ -- GitLab From 51434d4789d894a9f2aeb9c51fd8da9cbdb54872 Mon Sep 17 00:00:00 2001 From: Daya Adianto <dayaadianto@cs.ui.ac.id> Date: Tue, 3 Sep 2019 14:09:50 +0700 Subject: [PATCH 03/53] Add SonarQube Scanner configuration file --- sonar-project.properties | 13 +++++++++++++ 1 file changed, 13 insertions(+) create mode 100644 sonar-project.properties diff --git a/sonar-project.properties b/sonar-project.properties new file mode 100644 index 00000000..67cdf498 --- /dev/null +++ b/sonar-project.properties @@ -0,0 +1,13 @@ +# SonarQube Scanner Properties + +## Server +### sonar.host.url=[pass URL via CLI/CI] + +## Project Configuration +sonar.projectKey=id.ac.ui.cs.foss:kape +sonar.exclusions=/.devcontainer/,/.gitlab/ +sonar.scm.provider=git + +## Authentication +### sonar.login=[pass token via CLI/CI] + -- GitLab From df128b82de6c52c17f037191a092934e32bf3b26 Mon Sep 17 00:00:00 2001 From: Daya Adianto <dayaadianto@cs.ui.ac.id> Date: Tue, 3 Sep 2019 14:19:21 +0700 Subject: [PATCH 04/53] Update GitLab CI/CD configuration --- .gitlab-ci.yml | 91 ++++++++++++++++++++++++++------------------------ 1 file changed, 48 insertions(+), 43 deletions(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index d1e008c3..159f1664 100755 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -1,26 +1,29 @@ +--- +image: debian:jessie + test: + stage: test + before_script: + - apt-get update + - apt-get install -y sudo python-dev python-pip postgresql postgresql-client libpq-dev ligxss1 libappindicator1 libindicator7 + - wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | sudo apt-key add - + - sudo sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list' + - apt-get update + - apt-get install -y google-chrome-stable + - export CHROME_BIN=/usr/bin/google-chrome + - curl -sL https://deb.nodesource.com/setup_4.x | bash - + - apt-get install -y build-essential nodejs + - npm install npm -g script: - - apt-get install -f - - apt-get update -qy - - apt-get install -y python-dev python-pip sudo postgresql postgresql-client libpq-dev libxss1 libappindicator1 libindicator7 - - wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | sudo apt-key add - - - sudo sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list' - - apt-get update - - apt-get install -y google-chrome-stable - - export CHROME_BIN=/usr/bin/google-chrome - - curl -sL https://deb.nodesource.com/setup_4.x | sudo -E bash - - - sudo apt-get install -y nodejs - - sudo apt-get install -y build-essential - - npm install npm -g - - npm install - - npm run build-production - - service postgresql start - - sudo -u postgres psql -c "CREATE USER kape WITH PASSWORD 'kape' CREATEDB" - - sudo -u postgres psql -c "CREATE DATABASE kape OWNER kape" - - pip install -r requirements.txt - - python manage.py migrate - - python manage.py test - - npm run karma + - npm install + - npm run build-production + - service postgresql start + - sudo -u postgres psql -c "CREATE USER kape WITH PASSWORD 'kape' CREATEDB" + - sudo -u postgres psql -c "CREATE DATABASE kape OWNER kape" + - pip install -r requirements.txt + - python manage.py migrate + - python manage.py test + - npm run karma artifacts: paths: - test/ @@ -39,31 +42,33 @@ pages: staging: type: deploy script: - - apt-get update -qy - - apt-get install sshpass - - git clone https://gitlab.com/PPL2017csui/PPLA1.git - - cd PPLA1 - - git checkout develop - - git pull - - git remote add deployment ssh://kape@bot.recruit.id:8023/home/kape.git - - mkdir ~/.ssh - - echo -e "Host bot.recruit.id\n\tStrictHostKeyChecking no\n" >> ~/.ssh/config - - sshpass -p yukcarikape git push deployment develop:master --force + - apt-get update -qy + - apt-get install sshpass + - git clone https://gitlab.com/PPL2017csui/PPLA1.git + - cd PPLA1 + - git checkout develop + - git pull + - git remote add deployment ssh://kape@bot.recruit.id:8023/home/kape.git + - mkdir ~/.ssh + - echo -e "Host bot.recruit.id\n\tStrictHostKeyChecking no\n" >> ~/.ssh/config + - sshpass -p yukcarikape git push deployment develop:master --force only: - - develop + - develop production: type: deploy script: - - apt-get update -qy - - apt-get install sshpass - - git clone https://gitlab.com/PPL2017csui/PPLA1.git - - cd PPLA1 - - git checkout master - - git pull - - git remote add deployment ssh://kape@bot.recruit.id:8022/home/kape.git - - mkdir ~/.ssh - - echo -e "Host bot.recruit.id\n\tStrictHostKeyChecking no\n" >> ~/.ssh/config - - sshpass -p yukcarikape git push deployment master:master --force + - apt-get update -qy + - apt-get install sshpass + - git clone https://gitlab.com/PPL2017csui/PPLA1.git + - cd PPLA1 + - git checkout master + - git pull + - git remote add deployment ssh://kape@bot.recruit.id:8022/home/kape.git + - mkdir ~/.ssh + - echo -e "Host bot.recruit.id\n\tStrictHostKeyChecking no\n" >> ~/.ssh/config + - sshpass -p yukcarikape git push deployment master:master --force + when: manual only: - - master \ No newline at end of file + - master + -- GitLab From a6420e69f0180abc64f62af0b0cb6a3a9585746b Mon Sep 17 00:00:00 2001 From: Daya Adianto <dayaadianto@cs.ui.ac.id> Date: Tue, 3 Sep 2019 14:21:50 +0700 Subject: [PATCH 05/53] Fix typo --- .gitlab-ci.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 159f1664..87dd9b9e 100755 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -5,8 +5,8 @@ test: stage: test before_script: - apt-get update - - apt-get install -y sudo python-dev python-pip postgresql postgresql-client libpq-dev ligxss1 libappindicator1 libindicator7 - - wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | sudo apt-key add - + - apt-get install -y sudo python-dev python-pip postgresql postgresql-client libpq-dev lixss1 libappindicator1 libindicator7 + - wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | sudo apt-key add - - sudo sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list' - apt-get update - apt-get install -y google-chrome-stable -- GitLab From 2b7b2c903317f50e435d052d9af3eced96570cf8 Mon Sep 17 00:00:00 2001 From: Daya Adianto <dayaadianto@cs.ui.ac.id> Date: Tue, 3 Sep 2019 14:23:49 +0700 Subject: [PATCH 06/53] Fix typo --- .gitlab-ci.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 87dd9b9e..e98c36a1 100755 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -5,7 +5,7 @@ test: stage: test before_script: - apt-get update - - apt-get install -y sudo python-dev python-pip postgresql postgresql-client libpq-dev lixss1 libappindicator1 libindicator7 + - apt-get install -y sudo python-dev python-pip postgresql postgresql-client libpq-dev libxss1 libappindicator1 libindicator7 - wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | sudo apt-key add - - sudo sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list' - apt-get update -- GitLab From 534a8446b79e5bee062b9b1d1050e89fd2ff3e2b Mon Sep 17 00:00:00 2001 From: Daya Adianto <dayaadianto@cs.ui.ac.id> Date: Tue, 3 Sep 2019 14:30:57 +0700 Subject: [PATCH 07/53] State the project is licensed under ISC --- LICENSE | 16 ++++++++++++++++ README.md | 5 +++++ 2 files changed, 21 insertions(+) create mode 100644 LICENSE diff --git a/LICENSE b/LICENSE new file mode 100644 index 00000000..019f78f4 --- /dev/null +++ b/LICENSE @@ -0,0 +1,16 @@ +ISC License + +Copyright (c) 2017, PPLA1 Team & Faculty of Computer Science Universitas Indonesia + +Permission to use, copy, modify, and/or distribute this software for any +purpose with or without fee is hereby granted, provided that the above +copyright notice and this permission notice appear in all copies. + +THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES +WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF +MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR +ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES +WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN +ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF +OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + diff --git a/README.md b/README.md index 76cb0ca8..a2bffc9e 100755 --- a/README.md +++ b/README.md @@ -69,3 +69,8 @@ * Frontend test, run `npm run webpack` * Backend test, run `python manage.py test` + +## License + +Copyright (c) 2017, PPLA1 Team & Faculty of Computer Science Universitas +Indonesia. This project is licensed under [ISC](LICENSE) license. -- GitLab From 55349b7d5f9ca9f219a8e657ac18e2ff71bf5061 Mon Sep 17 00:00:00 2001 From: Daya Adianto <dayaadianto@cs.ui.ac.id> Date: Tue, 3 Sep 2019 14:31:55 +0700 Subject: [PATCH 08/53] Update GitLab CI/CD configuration --- .gitlab-ci.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index e98c36a1..5ad483a2 100755 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -5,7 +5,7 @@ test: stage: test before_script: - apt-get update - - apt-get install -y sudo python-dev python-pip postgresql postgresql-client libpq-dev libxss1 libappindicator1 libindicator7 + - apt-get install -y curl sudo python-dev python-pip postgresql postgresql-client libpq-dev libxss1 libappindicator1 libindicator7 wget - wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | sudo apt-key add - - sudo sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list' - apt-get update -- GitLab From 6833e7ba2aa95d5954a0df57a0a8eee68272402e Mon Sep 17 00:00:00 2001 From: Daya Adianto <dayaadianto@cs.ui.ac.id> Date: Tue, 3 Sep 2019 15:07:51 +0700 Subject: [PATCH 09/53] Add CI job to run SonarQube analysis --- .gitlab-ci.yml | 9 +++++++++ sonar-project.properties | 2 +- 2 files changed, 10 insertions(+), 1 deletion(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 5ad483a2..562a0edf 100755 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -28,6 +28,15 @@ test: paths: - test/ +SonarQube Analysis: + image: addianto/sonar-scanner-cli:latest + stage: test + script: + - sonar-scanner + -Dsonar.host.url=$SONARQUBE_HOST + -Dsonar.login=$SONARQUBE_TOKEN + - echo "TODO: Set this job to run only on `master` branch" + pages: stage: deploy dependencies: diff --git a/sonar-project.properties b/sonar-project.properties index 67cdf498..48a6c493 100644 --- a/sonar-project.properties +++ b/sonar-project.properties @@ -5,7 +5,7 @@ ## Project Configuration sonar.projectKey=id.ac.ui.cs.foss:kape -sonar.exclusions=/.devcontainer/,/.gitlab/ +sonar.exclusions=/.devcontainer/,/.gitlab/,*.config.js,/.tmp/, sonar.scm.provider=git ## Authentication -- GitLab From 5cfc789709f2d23a694ec8f501e64167a5d028c3 Mon Sep 17 00:00:00 2001 From: Daya Adianto <dayaadianto@cs.ui.ac.id> Date: Tue, 3 Sep 2019 15:10:47 +0700 Subject: [PATCH 10/53] Fix typo --- .gitlab-ci.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 562a0edf..58cb5e5f 100755 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -28,6 +28,7 @@ test: paths: - test/ +# TODO: Set the following CI job to only run on `master` branch SonarQube Analysis: image: addianto/sonar-scanner-cli:latest stage: test @@ -35,7 +36,6 @@ SonarQube Analysis: - sonar-scanner -Dsonar.host.url=$SONARQUBE_HOST -Dsonar.login=$SONARQUBE_TOKEN - - echo "TODO: Set this job to run only on `master` branch" pages: stage: deploy -- GitLab From c557ddf8083a39195d2fe09b8d92a2f7f136b79d Mon Sep 17 00:00:00 2001 From: Daya Adianto <dayaadianto@cs.ui.ac.id> Date: Tue, 3 Sep 2019 15:29:03 +0700 Subject: [PATCH 11/53] Revert to the original CI scripts --- .gitlab-ci.yml | 20 +++++++++++--------- 1 file changed, 11 insertions(+), 9 deletions(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 58cb5e5f..0cc21a00 100755 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -1,20 +1,23 @@ --- -image: debian:jessie +## We know it is outdated and not optimal, but the CI environment for this +## project was using ruby:2.1 container image +image: ruby:2.1 test: stage: test - before_script: - - apt-get update - - apt-get install -y curl sudo python-dev python-pip postgresql postgresql-client libpq-dev libxss1 libappindicator1 libindicator7 wget + script: + - apt-get install -f + - apt-get update -qy + - apt-get install -y python-dev python-pip sudo postgresql postgresql-client libpq-dev libxss1 libappindicator1 libindicator7 - wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | sudo apt-key add - - sudo sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list' - apt-get update - apt-get install -y google-chrome-stable - export CHROME_BIN=/usr/bin/google-chrome - - curl -sL https://deb.nodesource.com/setup_4.x | bash - - - apt-get install -y build-essential nodejs + - curl -sL https://deb.nodesource.com/setup_4.x | sudo -E bash - + - sudo apt-get install -y nodejs + - sudo apt-get install -y build-essential - npm install npm -g - script: - npm install - npm run build-production - service postgresql start @@ -28,7 +31,7 @@ test: paths: - test/ -# TODO: Set the following CI job to only run on `master` branch +## TODO: Set the following CI job to only run on `master` branch SonarQube Analysis: image: addianto/sonar-scanner-cli:latest stage: test @@ -80,4 +83,3 @@ production: when: manual only: - master - -- GitLab From 6217910e7a4160adb5592766a1041b7433e63d22 Mon Sep 17 00:00:00 2001 From: Daya Adianto <dayaadianto@cs.ui.ac.id> Date: Tue, 3 Sep 2019 16:15:47 +0700 Subject: [PATCH 12/53] Attempt to fix mismatched npm version --- .gitlab-ci.yml | 1 - 1 file changed, 1 deletion(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 0cc21a00..a7e0c368 100755 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -17,7 +17,6 @@ test: - curl -sL https://deb.nodesource.com/setup_4.x | sudo -E bash - - sudo apt-get install -y nodejs - sudo apt-get install -y build-essential - - npm install npm -g - npm install - npm run build-production - service postgresql start -- GitLab From b53540371067457c2743bff31580bace4d746321 Mon Sep 17 00:00:00 2001 From: Daya Adianto <dayaadianto@cs.ui.ac.id> Date: Tue, 3 Sep 2019 16:53:07 +0700 Subject: [PATCH 13/53] Pin versions on few required Python packages --- requirements.txt | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/requirements.txt b/requirements.txt index aeac0dc7..03301d7b 100755 --- a/requirements.txt +++ b/requirements.txt @@ -7,10 +7,10 @@ psycopg2==2.6.2 pyparsing==2.1.10 six==1.10.0 gunicorn -django-nose +django-nose==1.4.4 coverage django-rest-swagger django-silk requests -requests-mock -django-filter \ No newline at end of file +requests-mock==1.3.0 +django-filter -- GitLab From f653871c865854709fe7d34773a437ef746fbc60 Mon Sep 17 00:00:00 2001 From: Daya Adianto <dayaadianto@cs.ui.ac.id> Date: Tue, 3 Sep 2019 19:41:52 +0700 Subject: [PATCH 14/53] Pin django-filter version --- requirements.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/requirements.txt b/requirements.txt index 03301d7b..32f94175 100755 --- a/requirements.txt +++ b/requirements.txt @@ -13,4 +13,4 @@ django-rest-swagger django-silk requests requests-mock==1.3.0 -django-filter +django-filter==1.1.0 -- GitLab From f3454e4a6bf2428d58bc07e41d5f6a3212cf9cf1 Mon Sep 17 00:00:00 2001 From: Daya Adianto <dayaadianto@cs.ui.ac.id> Date: Tue, 3 Sep 2019 19:48:19 +0700 Subject: [PATCH 15/53] Pin django-silk version --- requirements.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/requirements.txt b/requirements.txt index 32f94175..46dd30dc 100755 --- a/requirements.txt +++ b/requirements.txt @@ -10,7 +10,7 @@ gunicorn django-nose==1.4.4 coverage django-rest-swagger -django-silk +django-silk==1.0.0 requests requests-mock==1.3.0 django-filter==1.1.0 -- GitLab From b63aacf7588fe51763bae82356b31b5af47e7aed Mon Sep 17 00:00:00 2001 From: Daya Adianto <dayaadianto@cs.ui.ac.id> Date: Tue, 3 Sep 2019 19:58:03 +0700 Subject: [PATCH 16/53] Update GitLab CI/CD configuration --- .gitlab-ci.yml | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index a7e0c368..72b4e5a2 100755 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -1,14 +1,11 @@ --- -## We know it is outdated and not optimal, but the CI environment for this -## project was using ruby:2.1 container image -image: ruby:2.1 +image: python:2-jessie test: stage: test script: - - apt-get install -f - apt-get update -qy - - apt-get install -y python-dev python-pip sudo postgresql postgresql-client libpq-dev libxss1 libappindicator1 libindicator7 + - apt-get install -y libappindicator1 libindicator7 libpq-dev libxss1 postgresql postgresql-client python-dev python-pip sudo \ - wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | sudo apt-key add - - sudo sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list' - apt-get update -- GitLab From bcfc860b74eeb4ad39d914a9a04357d4317f073e Mon Sep 17 00:00:00 2001 From: Daya Adianto <dayaadianto@cs.ui.ac.id> Date: Tue, 3 Sep 2019 20:06:20 +0700 Subject: [PATCH 17/53] Update GitLab CI/CD configuration --- .gitlab-ci.yml | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 72b4e5a2..c7502bf5 100755 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -4,12 +4,9 @@ image: python:2-jessie test: stage: test script: - - apt-get update -qy - - apt-get install -y libappindicator1 libindicator7 libpq-dev libxss1 postgresql postgresql-client python-dev python-pip sudo \ - wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | sudo apt-key add - - - sudo sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list' - - apt-get update - - apt-get install -y google-chrome-stable + - sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list' + - apt-get update && apt-get install -y google-chrome-stable libappindicator1 libindicator7 libpq-dev libxss1 postgresql postgresql-client python-dev python-pip sudo - export CHROME_BIN=/usr/bin/google-chrome - curl -sL https://deb.nodesource.com/setup_4.x | sudo -E bash - - sudo apt-get install -y nodejs -- GitLab From b1c3d693c00925b05364ae0b0621863842fde808 Mon Sep 17 00:00:00 2001 From: Daya Adianto <dayaadianto@cs.ui.ac.id> Date: Tue, 3 Sep 2019 20:06:53 +0700 Subject: [PATCH 18/53] Remove sudo in the first CI script --- .gitlab-ci.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index c7502bf5..c7f1bc55 100755 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -4,7 +4,7 @@ image: python:2-jessie test: stage: test script: - - wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | sudo apt-key add - + - wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - - sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list' - apt-get update && apt-get install -y google-chrome-stable libappindicator1 libindicator7 libpq-dev libxss1 postgresql postgresql-client python-dev python-pip sudo - export CHROME_BIN=/usr/bin/google-chrome -- GitLab From 04da53a372fb3e73afc5c492ee1d1e37d4e46446 Mon Sep 17 00:00:00 2001 From: Daya Adianto <dayaadianto@cs.ui.ac.id> Date: Wed, 11 Sep 2019 18:48:07 +0700 Subject: [PATCH 19/53] Use different DB when run on DEBUG or CI server --- README.md | 2 +- kape/settings.py | 25 +++++++++++++++++++++++++ 2 files changed, 26 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index a2bffc9e..6e6ec66c 100755 --- a/README.md +++ b/README.md @@ -72,5 +72,5 @@ ## License -Copyright (c) 2017, PPLA1 Team & Faculty of Computer Science Universitas +Copyright (c) 2017 PPLA1 Team & Faculty of Computer Science Universitas Indonesia. This project is licensed under [ISC](LICENSE) license. diff --git a/kape/settings.py b/kape/settings.py index 9b1643d0..eca5b4a5 100755 --- a/kape/settings.py +++ b/kape/settings.py @@ -20,6 +20,8 @@ SECRET_KEY = 'gz!k*@!n8h$yny1)zp!e5#w8!s4%*wqnur5$qnr@$*xx_o+aij' # SECURITY WARNING: don't run with debug turned on in production! DEBUG = (len(sys.argv) > 1 and sys.argv[1] == 'runserver') +CI_SERVER = os.getenv('CI_SERVER', 'no') == 'yes' + ALLOWED_HOSTS = [u'bot.recruit.id',u'kape.recruit.id',u'104.236.76.161',u'localhost',u'127.0.0.1'] @@ -99,6 +101,29 @@ DATABASES = { } } +TEST_DATABASE = { + 'ENGINE': 'django.db.backends.postgresql_psycopg2', + 'NAME': 'kape', + 'USER': 'kape', + 'PASSWORD': 'kape', + 'HOST': 'localhost', + 'PORT': '5432', +} + +CI_DATABASE = { + 'ENGINE': 'django.db.backends.postgresql_psycopg2', + 'NAME': 'kape', + 'USER': 'kape', + 'PASSWORD': 'kape', + 'HOST': 'postgres', + 'PORT': '5432', +} + +if DEBUG: + DATABASES['default'] = TEST_DATABASE + +if CI_SERVER == 'yes': + DATABASES['default'] = CI_DATABASE # Password validation # https://docs.djangoproject.com/en/1.10/ref/settings/#auth-password-validators -- GitLab From d82a67c4925d078cf148f2bf71da388970c250e9 Mon Sep 17 00:00:00 2001 From: Daya Adianto <dayaadianto@cs.ui.ac.id> Date: Thu, 12 Sep 2019 07:18:41 +0700 Subject: [PATCH 20/53] Add Postgres service into the CI pipeline --- .gitlab-ci.yml | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index c7f1bc55..3878a7c3 100755 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -2,6 +2,8 @@ image: python:2-jessie test: + services: + - postgres:9.6-alpine stage: test script: - wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - @@ -9,8 +11,7 @@ test: - apt-get update && apt-get install -y google-chrome-stable libappindicator1 libindicator7 libpq-dev libxss1 postgresql postgresql-client python-dev python-pip sudo - export CHROME_BIN=/usr/bin/google-chrome - curl -sL https://deb.nodesource.com/setup_4.x | sudo -E bash - - - sudo apt-get install -y nodejs - - sudo apt-get install -y build-essential + - apt-get install -y build-essential nodejs - npm install - npm run build-production - service postgresql start -- GitLab From 9fda96f57e69a8944296a069e0abe5b2028ae888 Mon Sep 17 00:00:00 2001 From: Daya Adianto <dayaadianto@cs.ui.ac.id> Date: Thu, 12 Sep 2019 07:27:18 +0700 Subject: [PATCH 21/53] Run PostgreSQL DB as a separate service in the CI server --- .gitlab-ci.yml | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 3878a7c3..6abd33ea 100755 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -4,19 +4,20 @@ image: python:2-jessie test: services: - postgres:9.6-alpine + variables: + POSTGRES_DB: kape + POSTGRES_USER: kape + POSTGRES_PASSWORD: kape stage: test script: - wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - - sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list' - - apt-get update && apt-get install -y google-chrome-stable libappindicator1 libindicator7 libpq-dev libxss1 postgresql postgresql-client python-dev python-pip sudo + - apt-get update && apt-get install -y google-chrome-stable libappindicator1 libindicator7 libpq-dev libxss1 python-dev python-pip sudo - export CHROME_BIN=/usr/bin/google-chrome - curl -sL https://deb.nodesource.com/setup_4.x | sudo -E bash - - apt-get install -y build-essential nodejs - npm install - npm run build-production - - service postgresql start - - sudo -u postgres psql -c "CREATE USER kape WITH PASSWORD 'kape' CREATEDB" - - sudo -u postgres psql -c "CREATE DATABASE kape OWNER kape" - pip install -r requirements.txt - python manage.py migrate - python manage.py test -- GitLab From c38e19be1c97035a56582f98dcd419ebcb2a7b80 Mon Sep 17 00:00:00 2001 From: Daya Adianto <dayaadianto@cs.ui.ac.id> Date: Thu, 12 Sep 2019 07:55:53 +0700 Subject: [PATCH 22/53] Debug DB service running on CI --- .gitlab-ci.yml | 3 +++ kape/settings.py | 4 +--- 2 files changed, 4 insertions(+), 3 deletions(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 6abd33ea..b9d1c0e8 100755 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -9,6 +9,9 @@ test: POSTGRES_USER: kape POSTGRES_PASSWORD: kape stage: test + before_script: + - echo $CI + - echo $CI_SERVER script: - wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - - sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list' diff --git a/kape/settings.py b/kape/settings.py index eca5b4a5..e7c22026 100755 --- a/kape/settings.py +++ b/kape/settings.py @@ -20,8 +20,6 @@ SECRET_KEY = 'gz!k*@!n8h$yny1)zp!e5#w8!s4%*wqnur5$qnr@$*xx_o+aij' # SECURITY WARNING: don't run with debug turned on in production! DEBUG = (len(sys.argv) > 1 and sys.argv[1] == 'runserver') -CI_SERVER = os.getenv('CI_SERVER', 'no') == 'yes' - ALLOWED_HOSTS = [u'bot.recruit.id',u'kape.recruit.id',u'104.236.76.161',u'localhost',u'127.0.0.1'] @@ -122,7 +120,7 @@ CI_DATABASE = { if DEBUG: DATABASES['default'] = TEST_DATABASE -if CI_SERVER == 'yes': +if os.getenv('CI_SERVER') == 'yes': DATABASES['default'] = CI_DATABASE # Password validation -- GitLab From 5ecf4f7aa642dc646c40aad047a49472151eabb0 Mon Sep 17 00:00:00 2001 From: Daya Adianto <dayaadianto@cs.ui.ac.id> Date: Thu, 12 Sep 2019 10:05:27 +0700 Subject: [PATCH 23/53] Remove debug log in the CI job --- .gitlab-ci.yml | 3 --- 1 file changed, 3 deletions(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index b9d1c0e8..6abd33ea 100755 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -9,9 +9,6 @@ test: POSTGRES_USER: kape POSTGRES_PASSWORD: kape stage: test - before_script: - - echo $CI - - echo $CI_SERVER script: - wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - - sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list' -- GitLab From 00ae290cdc69f90dd65c4ceb37a9099e942c2671 Mon Sep 17 00:00:00 2001 From: Daya Adianto <dayaadianto@cs.ui.ac.id> Date: Thu, 12 Sep 2019 10:06:21 +0700 Subject: [PATCH 24/53] Set SonarScanner job to run only on master --- .gitlab-ci.yml | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 6abd33ea..56c46956 100755 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -26,14 +26,15 @@ test: paths: - test/ -## TODO: Set the following CI job to only run on `master` branch -SonarQube Analysis: +SonarScanner Analysis: image: addianto/sonar-scanner-cli:latest stage: test script: - sonar-scanner -Dsonar.host.url=$SONARQUBE_HOST -Dsonar.login=$SONARQUBE_TOKEN + only: + - master pages: stage: deploy -- GitLab From 0da29d187eded3f251076116426511b1a6d3cbd2 Mon Sep 17 00:00:00 2001 From: Daya Adianto <dayaadianto@cs.ui.ac.id> Date: Thu, 12 Sep 2019 10:33:43 +0700 Subject: [PATCH 25/53] Add VS Code Dev Container configuration --- .devcontainer/Dockerfile | 54 ++++++++++++++++++++++++++++++++ .devcontainer/devcontainer.json | 31 ++++++++++++++++++ .devcontainer/docker-compose.yml | 30 ++++++++++++++++++ 3 files changed, 115 insertions(+) create mode 100644 .devcontainer/Dockerfile create mode 100644 .devcontainer/devcontainer.json create mode 100644 .devcontainer/docker-compose.yml diff --git a/.devcontainer/Dockerfile b/.devcontainer/Dockerfile new file mode 100644 index 00000000..28488893 --- /dev/null +++ b/.devcontainer/Dockerfile @@ -0,0 +1,54 @@ +#------------------------------------------------------------------------------------------------------------- +# Copyright (c) Microsoft Corporation. All rights reserved. +# Licensed under the MIT License. See https://go.microsoft.com/fwlink/?linkid=2090316 for license information. +#------------------------------------------------------------------------------------------------------------- + +FROM python:2-jessie + +# Avoid warnings by switching to noninteractive +ENV DEBIAN_FRONTEND=noninteractive + +ENV PYTHONUNBUFFERED 1 + +# This Dockerfile adds a non-root 'vscode' user with sudo access. However, for Linux, +# this user's GID/UID must match your local user UID/GID to avoid permission issues +# with bind mounts. Update USER_UID / USER_GID if yours is not 1000. See +# https://aka.ms/vscode-remote/containers/non-root-user for details. +ARG USERNAME=vscode +ARG USER_UID=1000 +ARG USER_GID=$USER_UID + +# Uncomment the following COPY line and the corresponding lines in the `RUN` command if you wish to +# include your requirements in the image itself. It is suggested that you only do this if your +# requirements rarely (if ever) change. +COPY requirements.txt /tmp/pip-tmp/ + +# Configure apt and install packages +RUN apt-get update \ + && apt-get -y install --no-install-recommends apt-utils dialog 2>&1 \ + # + # Verify git, process tools, lsb-release (common in install instructions for CLIs) installed + && apt-get -y install git iproute2 procps lsb-release \ + # + # Install pylint + && pip install pylint \ + # + # Update Python environment based on requirements.txt + && pip --disable-pip-version-check --no-cache-dir install -r /tmp/pip-tmp/requirements.txt \ + && rm -rf /tmp/pip-tmp \ + # + # Create a non-root user to use if preferred - see https://aka.ms/vscode-remote/containers/non-root-user. + && groupadd --gid $USER_GID $USERNAME \ + && useradd -s /bin/bash --uid $USER_UID --gid $USER_GID -m $USERNAME \ + # [Optional] Add sudo support for the non-root user + && apt-get install -y sudo \ + && echo $USERNAME ALL=\(root\) NOPASSWD:ALL > /etc/sudoers.d/$USERNAME\ + && chmod 0440 /etc/sudoers.d/$USERNAME \ + # + # Clean up + && apt-get autoremove -y \ + && apt-get clean -y \ + && rm -rf /var/lib/apt/lists/* + +# Switch back to dialog for any ad-hoc use of apt-get +ENV DEBIAN_FRONTEND= diff --git a/.devcontainer/devcontainer.json b/.devcontainer/devcontainer.json new file mode 100644 index 00000000..cf43fde5 --- /dev/null +++ b/.devcontainer/devcontainer.json @@ -0,0 +1,31 @@ +// If you want to run as a non-root user in the container, see .devcontainer/docker-compose.yml. +{ + "name": "Python 2 & PostgreSQL", + "dockerComposeFile": "docker-compose.yml", + "service": "app", + "workspaceFolder": "/workspace", + + // Use 'settings' to set *default* container specific settings.json values on container create. + // You can edit these settings after create using File > Preferences > Settings > Remote. + "settings": { + "terminal.integrated.shell.linux": "/bin/bash", + "python.pythonPath": "/usr/local/bin/python", + "python.linting.enabled": true, + "python.linting.pylintEnabled": true, + "python.linting.pylintPath": "/usr/local/bin/pylint" + }, + + // Uncomment the next line if you want start specific services in your Docker Compose config. + // "runServices": [], + + // Uncomment the next line if you want to keep your containers running after VS Code shuts down. + // "shutdownAction": "none", + + // Uncomment the next line to run commands after the container is created. + // "postCreateCommand": "pip install -r requirements.txt", + + // Add the IDs of extensions you want installed when the container is created in the array below. + "extensions": [ + "ms-python.python" + ] +} diff --git a/.devcontainer/docker-compose.yml b/.devcontainer/docker-compose.yml new file mode 100644 index 00000000..74ce3920 --- /dev/null +++ b/.devcontainer/docker-compose.yml @@ -0,0 +1,30 @@ +version: '3' + +services: + app: + # Uncomment the next line to use a non-root user. On Linux, this will prevent + # new files getting created as root, but you may need to update the USER_UID + # and USER_GID in .devcontainer/Dockerfile to match your user if not 1000. + user: vscode + + build: + context: .. + dockerfile: .devcontainer/Dockerfile + + volumes: + - ..:/workspace + + # Overrides default command so things don't shut down after the process ends. + command: sleep infinity + + links: + - db + db: + image: postgres:9.6 + restart: unless-stopped + ports: + - 5432:5432 + environment: + POSTGRES_DB: kape + POSTGRES_USER: kape + POSTGRES_PASSWORD: kape -- GitLab From 9573bd1dcebda10c6f7cbc6bcfa9ccb00cc405b8 Mon Sep 17 00:00:00 2001 From: Daya Adianto <dayaadianto@cs.ui.ac.id> Date: Thu, 12 Sep 2019 15:59:53 +0700 Subject: [PATCH 26/53] Create Dockerfile --- Dockerfile | 50 ++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 50 insertions(+) create mode 100644 Dockerfile diff --git a/Dockerfile b/Dockerfile new file mode 100644 index 00000000..2ecd354c --- /dev/null +++ b/Dockerfile @@ -0,0 +1,50 @@ +FROM python:2-jessie + +# Avoid warnings by switching to noninteractive +ENV DEBIAN_FRONTEND=noninteractive +ENV PYTHONUNBUFFERED 1 + +ARG USERNAME=kape +ARG USER_UID=1000 +ARG USER_GID=$USER_UID + +COPY requirements.txt /tmp/pip-tmp/ + +# Install system-level dependencies +RUN apt-get update && apt-get -y install --no-install-recommends \ + libappindicator1 \ + libindicator7 \ + libpq-dev \ + libxss1 +RUN curl -sL https://deb.nodesource.com/setup_4.x | bash - \ + && apt-get update && apt-get -y install --no-install-recommends \ + build-essential \ + nodejs +RUN pip --disable-pip-version-check --no-cache-dir install -r /tmp/pip-tmp/requirements.txt \ + && rm -rf /tmp/pip-tmp +RUN apt-get -y autoremove \ + && apt-get -y clean \ + && rm -rf /var/lib/apt/lists/* + +# Setup user +RUN groupadd --gid ${USER_GID} ${USERNAME} \ + && useradd -s /bin/bash --uid ${USER_UID} --gid ${USER_GID} -m ${USERNAME} +WORKDIR /home/kape + +COPY . app +RUN cd app \ + && npm install \ + && npm run build-production \ + && python manage.py collectstatic --noinput \ + && cd .. + +# Switch back to dialog for any ad-hoc use of apt-get +ENV DEBIAN_FRONTEND= + +# Expose required ports +EXPOSE 8001 + +# Run the app as non-root user +USER kape +WORKDIR /home/kape/app +CMD ["gunicorn", "--bind", "0.0.0.0:8001", "kape.wsgi:application", "--reload"] \ No newline at end of file -- GitLab From a76a6935336a66b080f5647d4ebfaa2f0654bc48 Mon Sep 17 00:00:00 2001 From: Daya Adianto <dayaadianto@cs.ui.ac.id> Date: Thu, 12 Sep 2019 16:47:13 +0700 Subject: [PATCH 27/53] Load DB configuration from environment variables --- Dockerfile | 5 +++++ LICENSE | 3 ++- kape/settings.py | 10 +++++----- 3 files changed, 12 insertions(+), 6 deletions(-) diff --git a/Dockerfile b/Dockerfile index 2ecd354c..d2a3a5b2 100644 --- a/Dockerfile +++ b/Dockerfile @@ -47,4 +47,9 @@ EXPOSE 8001 # Run the app as non-root user USER kape WORKDIR /home/kape/app +ENV KAPE_DB_NAME=kape +ENV KAPE_DB_USER=kape +ENV KAPE_DB_PASSWORD=kape +ENV KAPE_DB_HOST=db +ENV KAPE_DB_PORT=5432 CMD ["gunicorn", "--bind", "0.0.0.0:8001", "kape.wsgi:application", "--reload"] \ No newline at end of file diff --git a/LICENSE b/LICENSE index 019f78f4..47c70f83 100644 --- a/LICENSE +++ b/LICENSE @@ -1,6 +1,7 @@ ISC License -Copyright (c) 2017, PPLA1 Team & Faculty of Computer Science Universitas Indonesia +Copyright (c) 2017 PPLA1 Team & Faculty of Computer Science Universitas +Indonesia Permission to use, copy, modify, and/or distribute this software for any purpose with or without fee is hereby granted, provided that the above diff --git a/kape/settings.py b/kape/settings.py index e7c22026..0d66d794 100755 --- a/kape/settings.py +++ b/kape/settings.py @@ -91,11 +91,11 @@ WSGI_APPLICATION = 'kape.wsgi.application' DATABASES = { 'default': { 'ENGINE': 'django.db.backends.postgresql_psycopg2', - 'NAME': 'kape', - 'USER': 'kape', - 'PASSWORD': 'kape', - 'HOST': 'localhost', - 'PORT': '', + 'NAME': os.getenv('KAPE_DB_NAME', 'kape'), + 'USER': os.getenv('KAPE_DB_USER', 'kape'), + 'PASSWORD': os.getenv('KAPE_DB_PASSWORD', 'kape'), + 'HOST': os.getenv('KAPE_DB_HOST', 'localhost'), + 'PORT': os.getenv('KAPE_DB_PORT', ''), } } -- GitLab From 76fc1ff9cfd749fae55e57e085256098cbc1b9c7 Mon Sep 17 00:00:00 2001 From: Daya Adianto <dayaadianto@cs.ui.ac.id> Date: Thu, 12 Sep 2019 16:47:29 +0700 Subject: [PATCH 28/53] Create .dockerignore --- .dockerignore | 329 ++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 329 insertions(+) create mode 100644 .dockerignore diff --git a/.dockerignore b/.dockerignore new file mode 100644 index 00000000..f7a08602 --- /dev/null +++ b/.dockerignore @@ -0,0 +1,329 @@ +# Created by https://www.gitignore.io/api/node,python,pycharm+all,visualstudiocode +# Edit at https://www.gitignore.io/?templates=node,python,pycharm+all,visualstudiocode + +### Node ### +# Logs +logs +*.log +npm-debug.log* +yarn-debug.log* +yarn-error.log* +lerna-debug.log* + +# Diagnostic reports (https://nodejs.org/api/report.html) +report.[0-9]*.[0-9]*.[0-9]*.[0-9]*.json + +# Runtime data +pids +*.pid +*.seed +*.pid.lock + +# Directory for instrumented libs generated by jscoverage/JSCover +lib-cov + +# Coverage directory used by tools like istanbul +coverage +*.lcov + +# nyc test coverage +.nyc_output + +# Grunt intermediate storage (https://gruntjs.com/creating-plugins#storing-task-files) +.grunt + +# Bower dependency directory (https://bower.io/) +bower_components + +# node-waf configuration +.lock-wscript + +# Compiled binary addons (https://nodejs.org/api/addons.html) +build/Release + +# Dependency directories +node_modules/ +jspm_packages/ + +# TypeScript v1 declaration files +typings/ + +# TypeScript cache +*.tsbuildinfo + +# Optional npm cache directory +.npm + +# Optional eslint cache +.eslintcache + +# Optional REPL history +.node_repl_history + +# Output of 'npm pack' +*.tgz + +# Yarn Integrity file +.yarn-integrity + +# dotenv environment variables file +.env +.env.test + +# parcel-bundler cache (https://parceljs.org/) +.cache + +# next.js build output +.next + +# nuxt.js build output +.nuxt + +# vuepress build output +.vuepress/dist + +# Serverless directories +.serverless/ + +# FuseBox cache +.fusebox/ + +# DynamoDB Local files +.dynamodb/ + +### PyCharm+all ### +# Covers JetBrains IDEs: IntelliJ, RubyMine, PhpStorm, AppCode, PyCharm, CLion, Android Studio and WebStorm +# Reference: https://intellij-support.jetbrains.com/hc/en-us/articles/206544839 + +# User-specific stuff +.idea/**/workspace.xml +.idea/**/tasks.xml +.idea/**/usage.statistics.xml +.idea/**/dictionaries +.idea/**/shelf + +# Generated files +.idea/**/contentModel.xml + +# Sensitive or high-churn files +.idea/**/dataSources/ +.idea/**/dataSources.ids +.idea/**/dataSources.local.xml +.idea/**/sqlDataSources.xml +.idea/**/dynamic.xml +.idea/**/uiDesigner.xml +.idea/**/dbnavigator.xml + +# Gradle +.idea/**/gradle.xml +.idea/**/libraries + +# Gradle and Maven with auto-import +# When using Gradle or Maven with auto-import, you should exclude module files, +# since they will be recreated, and may cause churn. Uncomment if using +# auto-import. +# .idea/modules.xml +# .idea/*.iml +# .idea/modules +# *.iml +# *.ipr + +# CMake +cmake-build-*/ + +# Mongo Explorer plugin +.idea/**/mongoSettings.xml + +# File-based project format +*.iws + +# IntelliJ +out/ + +# mpeltonen/sbt-idea plugin +.idea_modules/ + +# JIRA plugin +atlassian-ide-plugin.xml + +# Cursive Clojure plugin +.idea/replstate.xml + +# Crashlytics plugin (for Android Studio and IntelliJ) +com_crashlytics_export_strings.xml +crashlytics.properties +crashlytics-build.properties +fabric.properties + +# Editor-based Rest Client +.idea/httpRequests + +# Android studio 3.1+ serialized cache file +.idea/caches/build_file_checksums.ser + +### PyCharm+all Patch ### +# Ignores the whole .idea folder and all .iml files +# See https://github.com/joeblau/gitignore.io/issues/186 and https://github.com/joeblau/gitignore.io/issues/360 + +.idea/ + +# Reason: https://github.com/joeblau/gitignore.io/issues/186#issuecomment-249601023 + +*.iml +modules.xml +.idea/misc.xml +*.ipr + +# Sonarlint plugin +.idea/sonarlint + +### Python ### +# Byte-compiled / optimized / DLL files +__pycache__/ +*.py[cod] +*$py.class + +# C extensions +*.so + +# Distribution / packaging +.Python +build/ +develop-eggs/ +dist/ +downloads/ +eggs/ +.eggs/ +lib/ +lib64/ +parts/ +sdist/ +var/ +wheels/ +pip-wheel-metadata/ +share/python-wheels/ +*.egg-info/ +.installed.cfg +*.egg +MANIFEST + +# PyInstaller +# Usually these files are written by a python script from a template +# before PyInstaller builds the exe, so as to inject date/other infos into it. +*.manifest +*.spec + +# Installer logs +pip-log.txt +pip-delete-this-directory.txt + +# Unit test / coverage reports +htmlcov/ +.tox/ +.nox/ +.coverage +.coverage.* +nosetests.xml +coverage.xml +*.cover +.hypothesis/ +.pytest_cache/ + +# Translations +*.mo +*.pot + +# Django stuff: +local_settings.py +db.sqlite3 +db.sqlite3-journal + +# Flask stuff: +instance/ +.webassets-cache + +# Scrapy stuff: +.scrapy + +# Sphinx documentation +docs/_build/ + +# PyBuilder +target/ + +# Jupyter Notebook +.ipynb_checkpoints + +# IPython +profile_default/ +ipython_config.py + +# pyenv +.python-version + +# pipenv +# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control. +# However, in case of collaboration, if having platform-specific dependencies or dependencies +# having no cross-platform support, pipenv may install dependencies that don't work, or not +# install all needed dependencies. +#Pipfile.lock + +# celery beat schedule file +celerybeat-schedule + +# SageMath parsed files +*.sage.py + +# Environments +.venv +env/ +venv/ +ENV/ +env.bak/ +venv.bak/ + +# Spyder project settings +.spyderproject +.spyproject + +# Rope project settings +.ropeproject + +# mkdocs documentation +/site + +# mypy +.mypy_cache/ +.dmypy.json +dmypy.json + +# Pyre type checker +.pyre/ + +### VisualStudioCode ### +.vscode/* +!.vscode/settings.json +!.vscode/tasks.json +!.vscode/launch.json +!.vscode/extensions.json + +### VisualStudioCode Patch ### +# Ignore all local history of files +.history + +# End of https://www.gitignore.io/api/node,python,pycharm+all,visualstudiocode + +# Project-specific ignored files +.bash_history +webpack-stats.json +assets/bundles/* +test/* +.tmp/ + +# Docker-specific ignored files +.devcontainer +.tmp +.vscode +Dockerfile +LICENSE +README.md \ No newline at end of file -- GitLab From 9705bcc088607dfa54f52458eceaca86834017f0 Mon Sep 17 00:00:00 2001 From: Daya Adianto <dayaadianto@cs.ui.ac.id> Date: Thu, 12 Sep 2019 18:06:52 +0700 Subject: [PATCH 29/53] Clean seed data to sync with the models --- seeder.json | 3 --- 1 file changed, 3 deletions(-) diff --git a/seeder.json b/seeder.json index fb24bedc..0b34318e 100644 --- a/seeder.json +++ b/seeder.json @@ -191,7 +191,6 @@ "company1" ], "description": "Ini Company nomor satu", - "verified": true, "logo": "", "address": "Alamat company 1" } @@ -206,7 +205,6 @@ "company2" ], "description": "Ini Company nomor dua", - "verified": true, "logo": "", "address": "Ini alamat Company duar" } @@ -221,7 +219,6 @@ "Tutuplapak" ], "description": "Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nulla aliquet semper neque a fermentum. Duis ac tellus vitae augue iaculis ultrices. Curabitur commodo et neque nec feugiat. Morbi ac diam vel nunc commodo cursus. Phasellus nulla sapien, hendrerit vitae bibendum at, sollicitudin eu ante. Maecenas maximus, ante eu sollicitudin convallis, mauris nunc posuere risus, eu porttitor diam lacus vitae enim. Vestibulum ante ipsum primis in faucibus orci luctus et ultrices posuere cubilia Curae; Suspendisse at lectus a elit sollicitudin tempor. Nullam condimentum, justo nec tincidunt maximus, neque mi vulputate leo, sit amet lacinia massa ex eget sem. Duis ac erat facilisis, fringilla mauris in, consequat neque. In et neque consequat, vehicula magna at, efficitur ante. Mauris ac lacinia nibh.\r\n\r\nProin sagittis, lectus quis maximus varius, libero justo sollicitudin augue, non lacinia risus orci a enim. Curabitur iaculis enim quis ullamcorper commodo. Vivamus id nisi rhoncus, dignissim tellus quis, interdum est. Fusce sollicitudin eu libero ac feugiat. Vestibulum ante ipsum primis in faucibus orci luctus et ultrices posuere cubilia Curae; Maecenas semper posuere ex, sed accumsan libero iaculis faucibus. Fusce laoreet ac ligula ut consectetur. Donec tortor mauris, rutrum at sodales et, viverra in dolor. Sed bibendum elit et maximus volutpat. Phasellus justo ipsum, laoreet sit amet faucibus eu, ultricies suscipit mauris. Nullam aliquam libero eu ante ultrices mattis. Donec non justo hendrerit neque volutpat placerat. Ut euismod est nec sem mollis, sit amet porttitor massa rhoncus. Aenean id erat sit amet nunc ultrices scelerisque non in ipsum. Curabitur sollicitudin nulla id mi accumsan venenatis.", - "verified": true, "logo": "company-logo/8a258a48-3bce-4873-b5d1-538b360d0059.png", "address": "Jl. Kebayoran Baru nomor 13, Jakarta Barat" } -- GitLab From 14e0fe2755a853bc89ef05118bed085ddecd9e07 Mon Sep 17 00:00:00 2001 From: Daya Adianto <dayaadianto@cs.ui.ac.id> Date: Thu, 12 Sep 2019 18:35:37 +0700 Subject: [PATCH 30/53] Create docker-compose.yml --- docker-compose.yml | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) create mode 100644 docker-compose.yml diff --git a/docker-compose.yml b/docker-compose.yml new file mode 100644 index 00000000..27f7210a --- /dev/null +++ b/docker-compose.yml @@ -0,0 +1,22 @@ +--- +version: '3' + +services: + db: + image: postgres:9.6-alpine + environment: + POSTGRES_DB: kape + POSTGRES_USER: kape + POSTGRES_PASSWORD: kape + + app: + build: . + environment: + KAPE_DATABASE_HOST: db + volumes: + - .:/home/kape/app + ports: + - "8001" + user: kape + depends_on: + - db -- GitLab From 59238718dc21f6dd7783826ae23ace6c41002bc5 Mon Sep 17 00:00:00 2001 From: Daya Adianto <dayaadianto@cs.ui.ac.id> Date: Wed, 18 Sep 2019 22:57:41 +0700 Subject: [PATCH 31/53] Update Dockerfile --- Dockerfile | 17 ++++++++--------- 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/Dockerfile b/Dockerfile index d2a3a5b2..067b2331 100644 --- a/Dockerfile +++ b/Dockerfile @@ -29,27 +29,26 @@ RUN apt-get -y autoremove \ # Setup user RUN groupadd --gid ${USER_GID} ${USERNAME} \ && useradd -s /bin/bash --uid ${USER_UID} --gid ${USER_GID} -m ${USERNAME} -WORKDIR /home/kape -COPY . app -RUN cd app \ - && npm install \ +# Build frontend and backend (as root user) +WORKDIR /home/kape/app +COPY . . +RUN npm install \ && npm run build-production \ && python manage.py collectstatic --noinput \ - && cd .. + && chown -R ${USER_UID}:${USER_GID} /home/kape # Switch back to dialog for any ad-hoc use of apt-get ENV DEBIAN_FRONTEND= # Expose required ports EXPOSE 8001 - -# Run the app as non-root user -USER kape -WORKDIR /home/kape/app ENV KAPE_DB_NAME=kape ENV KAPE_DB_USER=kape ENV KAPE_DB_PASSWORD=kape ENV KAPE_DB_HOST=db ENV KAPE_DB_PORT=5432 + +# Run the app as non-root user +USER kape CMD ["gunicorn", "--bind", "0.0.0.0:8001", "kape.wsgi:application", "--reload"] \ No newline at end of file -- GitLab From f156892c375e797be6a8c169a904066efb65250c Mon Sep 17 00:00:00 2001 From: Daya Adianto <dayaadianto@cs.ui.ac.id> Date: Thu, 19 Sep 2019 15:40:46 +0700 Subject: [PATCH 32/53] Update Dockerfile --- Dockerfile | 1 + 1 file changed, 1 insertion(+) diff --git a/Dockerfile b/Dockerfile index 067b2331..282ac3b2 100644 --- a/Dockerfile +++ b/Dockerfile @@ -51,4 +51,5 @@ ENV KAPE_DB_PORT=5432 # Run the app as non-root user USER kape +VOLUME ["/home/kape/assets", "/home/kape/files"] CMD ["gunicorn", "--bind", "0.0.0.0:8001", "kape.wsgi:application", "--reload"] \ No newline at end of file -- GitLab From 9966ac523a9413c99cd18e064ffd959134cd050f Mon Sep 17 00:00:00 2001 From: Daya Adianto <dayaadianto@cs.ui.ac.id> Date: Thu, 19 Sep 2019 16:20:06 +0700 Subject: [PATCH 33/53] Update docker-compose.yml --- docker-compose.yml | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/docker-compose.yml b/docker-compose.yml index 27f7210a..2d86956b 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -10,13 +10,19 @@ services: POSTGRES_PASSWORD: kape app: - build: . + build: + context: . environment: KAPE_DATABASE_HOST: db volumes: - - .:/home/kape/app + - assets:/home/kape/assets + - files:/home/kape/files ports: - "8001" user: kape depends_on: - db + +volumes: + assets: + files: \ No newline at end of file -- GitLab From 09f5f7c7f15068ccc5055d69cf3753f686581c41 Mon Sep 17 00:00:00 2001 From: Daya Adianto <dayaadianto@cs.ui.ac.id> Date: Thu, 19 Sep 2019 16:20:48 +0700 Subject: [PATCH 34/53] Set static files and media path --- kape/settings.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/kape/settings.py b/kape/settings.py index 0d66d794..bbe07b28 100755 --- a/kape/settings.py +++ b/kape/settings.py @@ -160,9 +160,9 @@ USE_TZ = True # https://docs.djangoproject.com/en/1.10/howto/static-files/ STATIC_URL = '/assets/' -STATIC_ROOT = '/home/assets' +STATIC_ROOT = '/home/kape/assets' MEDIA_URL = '/files/' -MEDIA_ROOT = '/home/files' +MEDIA_ROOT = '/home/kape/files' REST_FRAMEWORK = { # Use Django's standard `django.contrib.auth` permissions, -- GitLab From 9740679b2a4c4fe0c3508907dcdf81f83e1d6e83 Mon Sep 17 00:00:00 2001 From: Daya Adianto <dayaadianto@cs.ui.ac.id> Date: Fri, 20 Sep 2019 16:35:08 +0700 Subject: [PATCH 35/53] Update app settings This commit updates the timezone and allowed hosts in the app. --- kape/settings.py | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-) diff --git a/kape/settings.py b/kape/settings.py index bbe07b28..7529209f 100755 --- a/kape/settings.py +++ b/kape/settings.py @@ -20,7 +20,8 @@ SECRET_KEY = 'gz!k*@!n8h$yny1)zp!e5#w8!s4%*wqnur5$qnr@$*xx_o+aij' # SECURITY WARNING: don't run with debug turned on in production! DEBUG = (len(sys.argv) > 1 and sys.argv[1] == 'runserver') -ALLOWED_HOSTS = [u'bot.recruit.id',u'kape.recruit.id',u'104.236.76.161',u'localhost',u'127.0.0.1'] +ALLOWED_HOSTS = [u'bot.recruit.id', u'kape.recruit.id', u'104.236.76.161', + u'localhost', u'127.0.0.1', u'pmpl.cs.ui.ac.id'] # Application definition @@ -146,13 +147,9 @@ AUTH_PASSWORD_VALIDATORS = [ # https://docs.djangoproject.com/en/1.10/topics/i18n/ LANGUAGE_CODE = 'en-us' - -TIME_ZONE = 'UTC' - +TIME_ZONE = 'Asia/Jakarta' USE_I18N = True - USE_L10N = True - USE_TZ = True -- GitLab From e5ece860587903afe5923a8d53a8d1df43652f9b Mon Sep 17 00:00:00 2001 From: Daya Adianto <dayaadianto@cs.ui.ac.id> Date: Fri, 20 Sep 2019 16:56:46 +0700 Subject: [PATCH 36/53] Finalise Dockerfile and docker-compose.yml --- .dockerignore | 1 + Dockerfile | 73 +++++++++++++++++++++++++++++++--------------- docker-compose.yml | 34 ++++++++++++++------- 3 files changed, 75 insertions(+), 33 deletions(-) diff --git a/.dockerignore b/.dockerignore index f7a08602..b047734a 100644 --- a/.dockerignore +++ b/.dockerignore @@ -325,5 +325,6 @@ test/* .tmp .vscode Dockerfile +docker-compose.yml LICENSE README.md \ No newline at end of file diff --git a/Dockerfile b/Dockerfile index 282ac3b2..4029b576 100644 --- a/Dockerfile +++ b/Dockerfile @@ -1,4 +1,11 @@ -FROM python:2-jessie +FROM node:4 AS frontend-builder + +WORKDIR /home/kape/app +COPY . . +RUN npm install \ + && npm run build-production + +FROM python:2-jessie AS app # Avoid warnings by switching to noninteractive ENV DEBIAN_FRONTEND=noninteractive @@ -10,39 +17,28 @@ ARG USER_GID=$USER_UID COPY requirements.txt /tmp/pip-tmp/ -# Install system-level dependencies -RUN apt-get update && apt-get -y install --no-install-recommends \ - libappindicator1 \ - libindicator7 \ - libpq-dev \ - libxss1 -RUN curl -sL https://deb.nodesource.com/setup_4.x | bash - \ - && apt-get update && apt-get -y install --no-install-recommends \ - build-essential \ - nodejs +# Install system packages RUN pip --disable-pip-version-check --no-cache-dir install -r /tmp/pip-tmp/requirements.txt \ && rm -rf /tmp/pip-tmp -RUN apt-get -y autoremove \ - && apt-get -y clean \ - && rm -rf /var/lib/apt/lists/* # Setup user RUN groupadd --gid ${USER_GID} ${USERNAME} \ && useradd -s /bin/bash --uid ${USER_UID} --gid ${USER_GID} -m ${USERNAME} -# Build frontend and backend (as root user) +# Build the app +USER kape WORKDIR /home/kape/app COPY . . -RUN npm install \ - && npm run build-production \ - && python manage.py collectstatic --noinput \ - && chown -R ${USER_UID}:${USER_GID} /home/kape +COPY --chown=${USERNAME} --from=frontend-builder /home/kape/app/assets/bundles ./assets/bundles +COPY --chown=${USERNAME} --from=frontend-builder /home/kape/app/webpack-stats.json ./ +RUN mkdir -p /home/kape/assets \ + && mkdir -p /home/kape/files \ + && python manage.py collectstatic --noinput # Switch back to dialog for any ad-hoc use of apt-get ENV DEBIAN_FRONTEND= -# Expose required ports -EXPOSE 8001 +# Set environment variables ENV KAPE_DB_NAME=kape ENV KAPE_DB_USER=kape ENV KAPE_DB_PASSWORD=kape @@ -50,6 +46,37 @@ ENV KAPE_DB_HOST=db ENV KAPE_DB_PORT=5432 # Run the app as non-root user -USER kape +EXPOSE 8001 VOLUME ["/home/kape/assets", "/home/kape/files"] -CMD ["gunicorn", "--bind", "0.0.0.0:8001", "kape.wsgi:application", "--reload"] \ No newline at end of file +CMD ["gunicorn", "--bind", "0.0.0.0:8001", "kape.wsgi:application", "--reload"] + +# Container image metadata +## Note to editors: metadata values for `created`, `version`, and `revision` +## keys must be provided during build process, i.e. `docker build` invocation. +## It is also possible to pass other metadata values via build arguments. +ARG IMAGE_CREATED="" +ARG IMAGE_AUTHORS="PPLA1 Team & Faculty of Computer Science Universitas Indonesia" +ARG IMAGE_SOURCE="https://gitlab.cs.ui.ac.id/foss/kape" +ARG IMAGE_VERSION="" +ARG IMAGE_REVISION="" +ARG IMAGE_VENDOR="Faculty of Computer Science Universitas Indonesia" +ARG IMAGE_TITLE="Kape" +ARG IMAGE_DESCRIPTION="Fork of https://gitlab.com/PPL2017csui/PPLA1" +LABEL org.opencontainers.image.created=${IMAGE_CREATED} \ + org.opencontainers.image.authors=${IMAGE_AUTHORS} \ + org.opencontainers.image.source=${IMAGE_SOURCE} \ + org.opencontainers.image.version=${IMAGE_VERSION} \ + org.opencontainers.image.revision=${IMAGE_REVISION} \ + org.opencontainers.image.vendor=${IMAGE_VENDOR} \ + org.opencontainers.image.licenses="ISC" \ + org.opencontainers.image.title=${IMAGE_TITLE} \ + org.opencontainers.image.description=${IMAGE_DESCRIPTION} + +## Note to editors: The following label assignments are to ensure backward +## compatibility with Label Schema standard +LABEL org.label-schema.build-date=${IMAGE_CREATED} \ + org.label-schema.vcs-url=${IMAGE_SOURCE} \ + org.label-schema.version=${IMAGE_VERSION} \ + org.label-schema.vendor=${IMAGE_VENDOR} \ + org.label-schema.title=${IMAGE_TITLE} \ + org.label-schema.description=${IMAGE_DESCRIPTION} \ No newline at end of file diff --git a/docker-compose.yml b/docker-compose.yml index 2d86956b..481a95bb 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -4,25 +4,39 @@ version: '3' services: db: image: postgres:9.6-alpine + expose: + - "5432" environment: - POSTGRES_DB: kape - POSTGRES_USER: kape - POSTGRES_PASSWORD: kape + POSTGRES_DB: ${KAPE_DB_NAME:-kape} + POSTGRES_USER: ${KAPE_DB_USER:-kape} + POSTGRES_PASSWORD: ${KAPE_DB_PASSWORD:-kape} app: build: context: . environment: - KAPE_DATABASE_HOST: db - volumes: - - assets:/home/kape/assets - - files:/home/kape/files - ports: + KAPE_DB_HOST: ${KAPE_DB_HOST:-db} + expose: - "8001" user: kape + volumes: + - assetsdata:/home/kape/assets:ro + - mediadata:/home/kape/files depends_on: - db + web: + image: nginx:1.16-alpine + ports: + - "80" + volumes: + - ./provision/nginx/nginx.conf:/etc/nginx/nginx.conf:ro + - ./provision/nginx/conf.d/:/etc/nginx/conf.d/:ro + - assetsdata:/var/www/assets:ro + - mediadata:/var/www/files:ro + depends_on: + - app + volumes: - assets: - files: \ No newline at end of file + assetsdata: + mediadata: -- GitLab From ad0c5fd2bab9f49d2bec604fee3b2cd257d9d59b Mon Sep 17 00:00:00 2001 From: Daya Adianto <dayaadianto@cs.ui.ac.id> Date: Fri, 20 Sep 2019 16:56:57 +0700 Subject: [PATCH 37/53] Add nginx configuration --- provision/nginx/conf.d/app.conf | 27 +++++++++++++++++++++++++++ provision/nginx/nginx.conf | 33 +++++++++++++++++++++++++++++++++ 2 files changed, 60 insertions(+) create mode 100644 provision/nginx/conf.d/app.conf create mode 100644 provision/nginx/nginx.conf diff --git a/provision/nginx/conf.d/app.conf b/provision/nginx/conf.d/app.conf new file mode 100644 index 00000000..24a63c24 --- /dev/null +++ b/provision/nginx/conf.d/app.conf @@ -0,0 +1,27 @@ +# nginx configuration for the app based on Gunicorn recommended settings. +upstream app_server { + server app:8001 fail_timeout=0; +} + +server { + listen 80 deferred; + listen [::]:80 deferred; + server_name localhost; + keepalive_timeout 5; + root /var/www; + + location / { + # Checks for static file, if not found proxy to app + try_files $uri @proxy_to_app; + } + + location @proxy_to_app { + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header X-Forwarded-Proto $scheme; + proxy_set_header Host $http_host; + # We don't want nginx trying to do something clever with redirects, + # we set the Host: header above already + proxy_redirect off; + proxy_pass http://app_server; + } +} \ No newline at end of file diff --git a/provision/nginx/nginx.conf b/provision/nginx/nginx.conf new file mode 100644 index 00000000..bbff48cb --- /dev/null +++ b/provision/nginx/nginx.conf @@ -0,0 +1,33 @@ +# Custom nginx configuration based on nginx:alpine and Gunicorn recommended +# settings. +user nginx; +worker_processes auto; + +error_log /var/log/nginx/error.log warn; +pid /var/run/nginx.pid; + +events { + worker_connections 1024; # Increase if you have lots of clients + accept_mutex on; # Set to 'on' if nginx worker_processes > 1 + use epoll; +} + +http { + include /etc/nginx/mime.types; + default_type application/octet-stream; + + log_format main '$remote_addr - $remote_user [$time_local] "$request" ' + '$status $body_bytes_sent "$http_referer" ' + '"$http_user_agent" "$http_x_forwarded_for"'; + + access_log /var/log/nginx/access.log main; + + sendfile on; + #tcp_nopush on; + + keepalive_timeout 65; + + #gzip on; + + include /etc/nginx/conf.d/*.conf; +} \ No newline at end of file -- GitLab From 83e9857bc71ffa6d8fd1ca7be9ebc1f6d22be610 Mon Sep 17 00:00:00 2001 From: Daya Adianto <dayaadianto@cs.ui.ac.id> Date: Fri, 20 Sep 2019 16:59:30 +0700 Subject: [PATCH 38/53] Remove VS Dev Container configuration files --- .devcontainer/Dockerfile | 54 -------------------------------- .devcontainer/devcontainer.json | 31 ------------------ .devcontainer/docker-compose.yml | 30 ------------------ 3 files changed, 115 deletions(-) delete mode 100644 .devcontainer/Dockerfile delete mode 100644 .devcontainer/devcontainer.json delete mode 100644 .devcontainer/docker-compose.yml diff --git a/.devcontainer/Dockerfile b/.devcontainer/Dockerfile deleted file mode 100644 index 28488893..00000000 --- a/.devcontainer/Dockerfile +++ /dev/null @@ -1,54 +0,0 @@ -#------------------------------------------------------------------------------------------------------------- -# Copyright (c) Microsoft Corporation. All rights reserved. -# Licensed under the MIT License. See https://go.microsoft.com/fwlink/?linkid=2090316 for license information. -#------------------------------------------------------------------------------------------------------------- - -FROM python:2-jessie - -# Avoid warnings by switching to noninteractive -ENV DEBIAN_FRONTEND=noninteractive - -ENV PYTHONUNBUFFERED 1 - -# This Dockerfile adds a non-root 'vscode' user with sudo access. However, for Linux, -# this user's GID/UID must match your local user UID/GID to avoid permission issues -# with bind mounts. Update USER_UID / USER_GID if yours is not 1000. See -# https://aka.ms/vscode-remote/containers/non-root-user for details. -ARG USERNAME=vscode -ARG USER_UID=1000 -ARG USER_GID=$USER_UID - -# Uncomment the following COPY line and the corresponding lines in the `RUN` command if you wish to -# include your requirements in the image itself. It is suggested that you only do this if your -# requirements rarely (if ever) change. -COPY requirements.txt /tmp/pip-tmp/ - -# Configure apt and install packages -RUN apt-get update \ - && apt-get -y install --no-install-recommends apt-utils dialog 2>&1 \ - # - # Verify git, process tools, lsb-release (common in install instructions for CLIs) installed - && apt-get -y install git iproute2 procps lsb-release \ - # - # Install pylint - && pip install pylint \ - # - # Update Python environment based on requirements.txt - && pip --disable-pip-version-check --no-cache-dir install -r /tmp/pip-tmp/requirements.txt \ - && rm -rf /tmp/pip-tmp \ - # - # Create a non-root user to use if preferred - see https://aka.ms/vscode-remote/containers/non-root-user. - && groupadd --gid $USER_GID $USERNAME \ - && useradd -s /bin/bash --uid $USER_UID --gid $USER_GID -m $USERNAME \ - # [Optional] Add sudo support for the non-root user - && apt-get install -y sudo \ - && echo $USERNAME ALL=\(root\) NOPASSWD:ALL > /etc/sudoers.d/$USERNAME\ - && chmod 0440 /etc/sudoers.d/$USERNAME \ - # - # Clean up - && apt-get autoremove -y \ - && apt-get clean -y \ - && rm -rf /var/lib/apt/lists/* - -# Switch back to dialog for any ad-hoc use of apt-get -ENV DEBIAN_FRONTEND= diff --git a/.devcontainer/devcontainer.json b/.devcontainer/devcontainer.json deleted file mode 100644 index cf43fde5..00000000 --- a/.devcontainer/devcontainer.json +++ /dev/null @@ -1,31 +0,0 @@ -// If you want to run as a non-root user in the container, see .devcontainer/docker-compose.yml. -{ - "name": "Python 2 & PostgreSQL", - "dockerComposeFile": "docker-compose.yml", - "service": "app", - "workspaceFolder": "/workspace", - - // Use 'settings' to set *default* container specific settings.json values on container create. - // You can edit these settings after create using File > Preferences > Settings > Remote. - "settings": { - "terminal.integrated.shell.linux": "/bin/bash", - "python.pythonPath": "/usr/local/bin/python", - "python.linting.enabled": true, - "python.linting.pylintEnabled": true, - "python.linting.pylintPath": "/usr/local/bin/pylint" - }, - - // Uncomment the next line if you want start specific services in your Docker Compose config. - // "runServices": [], - - // Uncomment the next line if you want to keep your containers running after VS Code shuts down. - // "shutdownAction": "none", - - // Uncomment the next line to run commands after the container is created. - // "postCreateCommand": "pip install -r requirements.txt", - - // Add the IDs of extensions you want installed when the container is created in the array below. - "extensions": [ - "ms-python.python" - ] -} diff --git a/.devcontainer/docker-compose.yml b/.devcontainer/docker-compose.yml deleted file mode 100644 index 74ce3920..00000000 --- a/.devcontainer/docker-compose.yml +++ /dev/null @@ -1,30 +0,0 @@ -version: '3' - -services: - app: - # Uncomment the next line to use a non-root user. On Linux, this will prevent - # new files getting created as root, but you may need to update the USER_UID - # and USER_GID in .devcontainer/Dockerfile to match your user if not 1000. - user: vscode - - build: - context: .. - dockerfile: .devcontainer/Dockerfile - - volumes: - - ..:/workspace - - # Overrides default command so things don't shut down after the process ends. - command: sleep infinity - - links: - - db - db: - image: postgres:9.6 - restart: unless-stopped - ports: - - 5432:5432 - environment: - POSTGRES_DB: kape - POSTGRES_USER: kape - POSTGRES_PASSWORD: kape -- GitLab From 2945271bbe81c1efb9e3f765623da502cb34a2d0 Mon Sep 17 00:00:00 2001 From: Daya Adianto <dayaadianto@cs.ui.ac.id> Date: Fri, 20 Sep 2019 17:14:14 +0700 Subject: [PATCH 39/53] Add CI job to build and publish the container image --- .gitlab-ci.yml | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 56c46956..be5177c7 100755 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -1,6 +1,11 @@ --- image: python:2-jessie +stages: + - build + - test + - deploy + test: services: - postgres:9.6-alpine @@ -79,3 +84,21 @@ production: when: manual only: - master + +Publish Container Image: + image: docker:19-git + stage: deploy + services: + - docker:19-dind + before_script: + - docker info + - docker login -u $DOCKERHUB_USERNAME -p $DOCKERHUB_PASSWORD + - echo "TODO Run this job only at `master` branch" + script: + - docker build + --build-arg=IMAGE_CREATED="$(date --utc -Iseconds)" + --build-arg=IMAGE_REVISION="$(git show --pretty=oneline | tac | tail -n 1 | awk '{print $1}')" + --tag $IMAGE_NAMESPACE/kape:latest . + - docker push $IMAGE_NAMESPACE/kape:latest + tags: + - docker -- GitLab From aa5cae3bdd2071738d488e6c5bb7084917615144 Mon Sep 17 00:00:00 2001 From: Daya Adianto <dayaadianto@cs.ui.ac.id> Date: Fri, 20 Sep 2019 17:53:51 +0700 Subject: [PATCH 40/53] Implement workaround in the GitLab CI/CD configuration --- .gitlab-ci.yml | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index be5177c7..ecac6eae 100755 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -89,7 +89,14 @@ Publish Container Image: image: docker:19-git stage: deploy services: - - docker:19-dind + # Workaround from https://forum.gitlab.com/t/docker-dind-stops-working-after-12-1-0-update/28664/2 + - name: docker:19-dind + entrypoint: ["env", "-u", "DOCKER_HOST"] + command: ["dockerd-entrypoint.sh"] + variables: + DOCKER_HOST: tcp://docker:2375/ + DOCKER_DRIVER: overlay2 + DOCKER_TLS_CERTDIR: "" before_script: - docker info - docker login -u $DOCKERHUB_USERNAME -p $DOCKERHUB_PASSWORD @@ -102,3 +109,4 @@ Publish Container Image: - docker push $IMAGE_NAMESPACE/kape:latest tags: - docker + allow_failure: true -- GitLab From 78099f72f751bb89543886071d456831e0f89a7d Mon Sep 17 00:00:00 2001 From: Daya Adianto <dayaadianto@cs.ui.ac.id> Date: Mon, 23 Sep 2019 13:44:47 +0700 Subject: [PATCH 41/53] Use proxy when building the container image in CSUI infra --- Dockerfile | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/Dockerfile b/Dockerfile index 4029b576..1fbc4b9a 100644 --- a/Dockerfile +++ b/Dockerfile @@ -2,7 +2,9 @@ FROM node:4 AS frontend-builder WORKDIR /home/kape/app COPY . . -RUN npm install \ +ARG NPM_PROXY="" +RUN test -n ${NPM_PROXY} && npm config set https-proxy ${NPM_PROXY}} \ + || npm install \ && npm run build-production FROM python:2-jessie AS app -- GitLab From da785b158c148711d772c7cea2a95274c71a3464 Mon Sep 17 00:00:00 2001 From: Daya Adianto <dayaadianto@cs.ui.ac.id> Date: Mon, 23 Sep 2019 13:45:12 +0700 Subject: [PATCH 42/53] Test the CI job for building container image --- .gitlab-ci.yml | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index ecac6eae..85454749 100755 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -87,7 +87,7 @@ production: Publish Container Image: image: docker:19-git - stage: deploy + stage: test services: # Workaround from https://forum.gitlab.com/t/docker-dind-stops-working-after-12-1-0-update/28664/2 - name: docker:19-dind @@ -100,7 +100,8 @@ Publish Container Image: before_script: - docker info - docker login -u $DOCKERHUB_USERNAME -p $DOCKERHUB_PASSWORD - - echo "TODO Run this job only at `master` branch" + - echo "TODO Run this job in 'deploy' stage" + - echo "TODO Run this job only at 'master' branch" script: - docker build --build-arg=IMAGE_CREATED="$(date --utc -Iseconds)" -- GitLab From 77ecb9928da5e37d2826456d8674316d678bbf29 Mon Sep 17 00:00:00 2001 From: Daya Adianto <dayaadianto@cs.ui.ac.id> Date: Mon, 23 Sep 2019 13:48:44 +0700 Subject: [PATCH 43/53] Fix variable replacement in the build argument --- Dockerfile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Dockerfile b/Dockerfile index 1fbc4b9a..1ace39cd 100644 --- a/Dockerfile +++ b/Dockerfile @@ -3,7 +3,7 @@ FROM node:4 AS frontend-builder WORKDIR /home/kape/app COPY . . ARG NPM_PROXY="" -RUN test -n ${NPM_PROXY} && npm config set https-proxy ${NPM_PROXY}} \ +RUN test -n $NPM_PROXY && npm config set https-proxy $NPM_PROXY \ || npm install \ && npm run build-production -- GitLab From f4d5d963e89621407771b59f3d53c7009331c684 Mon Sep 17 00:00:00 2001 From: Daya Adianto <dayaadianto@cs.ui.ac.id> Date: Mon, 23 Sep 2019 13:49:06 +0700 Subject: [PATCH 44/53] Ensure NPM_PROXY variable set during build process --- .gitlab-ci.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 85454749..0a843dc2 100755 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -104,6 +104,7 @@ Publish Container Image: - echo "TODO Run this job only at 'master' branch" script: - docker build + --build-arg=NPM_PROXY="https://proxy.cs.ui.ac.id:8080" --build-arg=IMAGE_CREATED="$(date --utc -Iseconds)" --build-arg=IMAGE_REVISION="$(git show --pretty=oneline | tac | tail -n 1 | awk '{print $1}')" --tag $IMAGE_NAMESPACE/kape:latest . -- GitLab From 8cc9f635a5d4109df8e91df08ea95fdd84e3b20f Mon Sep 17 00:00:00 2001 From: Daya Adianto <dayaadianto@cs.ui.ac.id> Date: Mon, 23 Sep 2019 13:52:14 +0700 Subject: [PATCH 45/53] Fix Dockerfile --- Dockerfile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Dockerfile b/Dockerfile index 1ace39cd..839ac80e 100644 --- a/Dockerfile +++ b/Dockerfile @@ -3,7 +3,7 @@ FROM node:4 AS frontend-builder WORKDIR /home/kape/app COPY . . ARG NPM_PROXY="" -RUN test -n $NPM_PROXY && npm config set https-proxy $NPM_PROXY \ +RUN test -n ${NPM_PROXY} && npm config set https-proxy ${NPM_PROXY} \ || npm install \ && npm run build-production -- GitLab From f871813729c9f5e679b3564ca57776030de3ef27 Mon Sep 17 00:00:00 2001 From: Daya Adianto <dayaadianto@cs.ui.ac.id> Date: Mon, 23 Sep 2019 13:54:33 +0700 Subject: [PATCH 46/53] Fix Dockerfile --- Dockerfile | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Dockerfile b/Dockerfile index 839ac80e..f80bf6e0 100644 --- a/Dockerfile +++ b/Dockerfile @@ -3,8 +3,8 @@ FROM node:4 AS frontend-builder WORKDIR /home/kape/app COPY . . ARG NPM_PROXY="" -RUN test -n ${NPM_PROXY} && npm config set https-proxy ${NPM_PROXY} \ - || npm install \ +RUN npm config set https-proxy ${NPM_PROXY} \ + && npm install \ && npm run build-production FROM python:2-jessie AS app -- GitLab From 235c460656a90baedf63d58bb794b8957e33c1e9 Mon Sep 17 00:00:00 2001 From: Daya Adianto <dayaadianto@cs.ui.ac.id> Date: Mon, 23 Sep 2019 14:00:24 +0700 Subject: [PATCH 47/53] Fix Dockerfile --- Dockerfile | 1 + 1 file changed, 1 insertion(+) diff --git a/Dockerfile b/Dockerfile index f80bf6e0..306e9ed4 100644 --- a/Dockerfile +++ b/Dockerfile @@ -4,6 +4,7 @@ WORKDIR /home/kape/app COPY . . ARG NPM_PROXY="" RUN npm config set https-proxy ${NPM_PROXY} \ + && npm config set proxy ${NPM_PROXY} \ && npm install \ && npm run build-production -- GitLab From 678ff314f6a953e2634edf909a18459d895afec3 Mon Sep 17 00:00:00 2001 From: Daya Adianto <dayaadianto@cs.ui.ac.id> Date: Mon, 23 Sep 2019 14:12:23 +0700 Subject: [PATCH 48/53] Fix Dockerfile --- Dockerfile | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/Dockerfile b/Dockerfile index 306e9ed4..f2c6d104 100644 --- a/Dockerfile +++ b/Dockerfile @@ -3,8 +3,7 @@ FROM node:4 AS frontend-builder WORKDIR /home/kape/app COPY . . ARG NPM_PROXY="" -RUN npm config set https-proxy ${NPM_PROXY} \ - && npm config set proxy ${NPM_PROXY} \ +RUN npm config set proxy ${NPM_PROXY} \ && npm install \ && npm run build-production -- GitLab From 8dbc1f2f0b24134bac4702bbc518fe71893ae0d3 Mon Sep 17 00:00:00 2001 From: Daya Adianto <dayaadianto@cs.ui.ac.id> Date: Mon, 23 Sep 2019 14:30:34 +0700 Subject: [PATCH 49/53] Debug Dockerfile --- Dockerfile | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/Dockerfile b/Dockerfile index f2c6d104..2129e8a0 100644 --- a/Dockerfile +++ b/Dockerfile @@ -3,7 +3,8 @@ FROM node:4 AS frontend-builder WORKDIR /home/kape/app COPY . . ARG NPM_PROXY="" -RUN npm config set proxy ${NPM_PROXY} \ +RUN echo ${NPM_PROXY} \ + && npm config set proxy ${NPM_PROXY} \ && npm install \ && npm run build-production -- GitLab From abad114b3523a493bbf43402fd51b76b612b2a61 Mon Sep 17 00:00:00 2001 From: Daya Adianto <dayaadianto@cs.ui.ac.id> Date: Mon, 23 Sep 2019 14:34:29 +0700 Subject: [PATCH 50/53] Fix GitLab CI/CD configuration --- .gitlab-ci.yml | 2 +- Dockerfile | 3 +-- 2 files changed, 2 insertions(+), 3 deletions(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 0a843dc2..9de6c551 100755 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -104,7 +104,7 @@ Publish Container Image: - echo "TODO Run this job only at 'master' branch" script: - docker build - --build-arg=NPM_PROXY="https://proxy.cs.ui.ac.id:8080" + --build-arg=NPM_PROXY=$NPM_PROXY --build-arg=IMAGE_CREATED="$(date --utc -Iseconds)" --build-arg=IMAGE_REVISION="$(git show --pretty=oneline | tac | tail -n 1 | awk '{print $1}')" --tag $IMAGE_NAMESPACE/kape:latest . diff --git a/Dockerfile b/Dockerfile index 2129e8a0..f2c6d104 100644 --- a/Dockerfile +++ b/Dockerfile @@ -3,8 +3,7 @@ FROM node:4 AS frontend-builder WORKDIR /home/kape/app COPY . . ARG NPM_PROXY="" -RUN echo ${NPM_PROXY} \ - && npm config set proxy ${NPM_PROXY} \ +RUN npm config set proxy ${NPM_PROXY} \ && npm install \ && npm run build-production -- GitLab From e68f121a1645ee02d427ccc07bdff99c5a0812f3 Mon Sep 17 00:00:00 2001 From: Daya Adianto <dayaadianto@cs.ui.ac.id> Date: Mon, 23 Sep 2019 15:09:35 +0700 Subject: [PATCH 51/53] Run container image building job only on master branch --- .gitlab-ci.yml | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 9de6c551..040e750d 100755 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -87,7 +87,7 @@ production: Publish Container Image: image: docker:19-git - stage: test + stage: deploy services: # Workaround from https://forum.gitlab.com/t/docker-dind-stops-working-after-12-1-0-update/28664/2 - name: docker:19-dind @@ -100,8 +100,6 @@ Publish Container Image: before_script: - docker info - docker login -u $DOCKERHUB_USERNAME -p $DOCKERHUB_PASSWORD - - echo "TODO Run this job in 'deploy' stage" - - echo "TODO Run this job only at 'master' branch" script: - docker build --build-arg=NPM_PROXY=$NPM_PROXY @@ -111,4 +109,7 @@ Publish Container Image: - docker push $IMAGE_NAMESPACE/kape:latest tags: - docker + retry: 1 allow_failure: true + only: + - master -- GitLab From 8ee8fe2953d8de09525685c14453656ac7f68485 Mon Sep 17 00:00:00 2001 From: Daya Adianto <dayaadianto@cs.ui.ac.id> Date: Tue, 24 Sep 2019 19:56:35 +0700 Subject: [PATCH 52/53] Update README --- README.md | 221 ++++++++++++++++++++++++++++++++++---------------- README.old.md | 71 ++++++++++++++++ 2 files changed, 221 insertions(+), 71 deletions(-) create mode 100644 README.old.md diff --git a/README.md b/README.md index 6e6ec66c..746c258b 100755 --- a/README.md +++ b/README.md @@ -1,74 +1,153 @@ -# How To Deploy to Production Server - -## Deployment -* Clone this repository -* Install docker, you can do this manually or using command `bash /provision/setup-docker.sh` -* Deploy using command `bash /provision/run-docker.sh DOCKERNAME APP_PORT SSH_PORT` for example `bash /provision/run-docker.sh staging 8000 8022` -* Profit :) - -## HTTPS Server and Domain Name -* Add your domain name on the `ALLOWED_HOSTS` field on `kape/settings.py` -* Add these server blocks on yout NGINX setting (change `DOMAIN_NAME` and `APP_PORT`) - ``` - server { - listen 80; - listen [::]:80; - server_name your.DOMAIN_NAME.com; - return 301 https://$server_name$request_uri; - } - - server { - listen 443 ssl default_server; - listen [::]:443 ssl default_server; - include snippets/ssl-your.DOMAIN_NAME.sslconfiguration.conf; - server_name your.DOMAIN_NAME.com; - location / { - proxy_pass http://localhost:APP_PORT; - proxy_set_header Host $host; - proxy_set_header X-Real-IP $remote_addr; - proxy_set_header X-Forwarded-Proto $scheme; - } - } - ``` - -## Git push on the production server -* You can push directly to production git if SSH port is not blocked -* In the repo directory, run `git remote add production ssh://kape@DOMAIN_NAME:SSH_PORT/home/kape.git` -* To push to production server, run `git push production master` (default docker ssh password : yukcarikape) - - -# How To Build for Development Environment - -## Requirements - -* NodeJS + Npm Package Manager -* Python + Pip Package Manager -* PostgreSQL - -## Installation - -* `git clone https://gitlab.com/PPL2017csui/PPLA1.git kape` -* `cd kape` -* `npm install` -* `pip install -r requirements.txt` -* `psql` insert these queries: - * `create user kape password 'kape';` to create user. - * `create database kape;` to create database. - * if you want to create another database user, please update the setting file `kape/settings.py` -* `python manage.py migrate` To migrate all the database - -## Run Development Mode App - -* `npm run webpack` to run webpack server. (if you don't want live update run `npm run build`, if you want the uglified+zipped+production ready version run `npm run build-production`) -* on a different terminal window, run `python manage.py runserver` to run API server - -### Now go to localhost:8000 and you'll see the App running! - - -## Run Unit Test - -* Frontend test, run `npm run webpack` -* Backend test, run `python manage.py test` +# Kanal Akses Pendaftaran KP Elektronik _(kape)_ + +> Internship matchmaking platform for students and companies. + +## Table of Contents + +- [Install](#install) +- [Running Development Mode (Classic)](#running-development-mode-classic) +- [Running Development Mode (Containerised)](#running-development-mode-containerised) +- [License](#license) + +## Install + +This project uses Python 2 and Node.js v4 for building the backend and frontend, +respectively. The backend uses Django Framework and PostgreSQL database, while +the frontend is developed using React. You need to install the required +dependencies prior to building and contributing to the project. + +- [Node.js v4.9.1](https://nodejs.org/en/download/releases/) and `npm` package + manager. + > Note: We recommend `nvm` (Node Version Manager) for installing Node.js. You + > can find `nvm` for your OS at the following links: + > - [nvm for macOS or GNU/Linux-based OS](https://github.com/creationix/nvm) + > - [nvm for Windows](https://github.com/coreybutler/nvm-windows) + > + > Once `nvm` has been installed, install Node.js and activate it by executing + > `nvm install 4.9.1` followed by `nvm activate 4.9.1`. +- [Python 2.7.16](https://www.python.org/downloads/release/) and `pip` package + manager. + > Note: We recommend using _virtual environment_ to isolate project-specific + > Python packages from system-level packages. You can install and use + > [`virtualenv`](https://virtualenv.pypa.io/en/stable/) package to create the + > virtual environment for this project. +- [PostgreSQL 9.6](https://www.postgresql.org/download/) and any database + client that can interface with PostgreSQL, e.g. `psql`, pgAdmin. + > Note: We recommend running the database as container or virtual machine to + > avoid cluttering your local development environment. Please consult the + > documentation for your container engine (e.g. Docker) or virtualization + > platform (e.g. VirtualBox or Hyper-V) on how to provision PostgreSQL + > database. If you had to use locally-installed database, make sure your + > own data will not interleave with the data generated by this project. + +Verify that Node.js and Python 2 have been successfully installed. Make sure +the interpreter for both platforms can be invoked from the shell. For example, +in `bash` shell (macOS or GNU/Linux-based OS): + +```bash +# Assuming there is a Python virtual environment directory named `env` in +# current path +$ source env/bin/activate +$ python --version # or: python2 --version +Python 2.7.16 +$ node --version +v4.9.1 +``` + +Now install the packages required by Node.js and Python 2: + +```bash +npm install +pip install -r requirements.txt +``` + +If you are cloning the project for the first time into your local development +environment, you need to create a new database user named `kape` and create a +new database with the same name for this project. Do not forget to set the +ownership of the new database to `kape` user. Once you have finished initalising +the database, perform database migration and seeding: + +```bash +python manage.py migrate +python manage.py loaddata seeder.json +``` + +> Note: Can't connect to the database? Adjust the database connection settings +> at [`settings.py`](kape/settings.py). + +Finally, verify that the test suites pass: + +```bash +# Run the test suite for backend +python manage.py test +``` + +To run the test suite for frontend, you need to build the frontend first before +running the test suite: + +```bash +npm run build-production +npm run karma +``` + +## Running Development Mode (Classic) + +To run the API (backend) server: + +```bash +python manage.py runserver +``` + +To serve the frontend: + +```bash +npm run webpack +# If you don't want live update, run: npm run build +# If you want the uglified, zipped, production-ready version, run: npm run build-production +``` + +You can see the app running by going to `localhost:8080` via your favourite +Web browser. + +## Running Development Mode (Containerised) + +If you have installed Docker and Docker Compose, you can build the app +into a container image and run it with container-based database and +Web server: + +```bash +docker-compose up --build --detach +docker-compose run --rm app python manage.py migrate +docker-compose run --rm app python manage.py loaddata seeder.json +``` + +> Explanation: +> 1. Rebuild the application (the frontend + backend), then start it along with +> the database (`postgres`) and Web server (`nginx`). +> 1. Perform database migration +> 1. Perform database seeding + +Before you can see the app, you need to know the port in the host that mapped +to the port in the Web server container: + +```bash +# The following is just an example. The output may vary in your environment. +$ docker-compose ps + Name Command State Ports +--------------------------------------------------------------------------- +kape_app_1 gunicorn --bind 0.0.0.0:80 ... Up 8001/tcp +kape_db_1 docker-entrypoint.sh postgres Up 5432/tcp +kape_web_1 nginx -g daemon off; Up 0.0.0.0:32770->80/tcp +``` + +In the above example, port 32770 is mapped to port 80 in the Web server +container. Thus, you can see the app via Web browser by opening +`http://localhost:32770`. + +> Note: Running the app and its dependencies as containers are convenient, but +> you will not be able to see any changes to the codebase in real-time. If +> you want to see changes in the codebase reflected in real-time, follow the +> classical approach for running the development mode. ## License diff --git a/README.old.md b/README.old.md new file mode 100644 index 00000000..d60f4bc9 --- /dev/null +++ b/README.old.md @@ -0,0 +1,71 @@ +# How To Deploy to Production Server + +## Deployment + +* Clone this repository +* Install docker, you can do this manually or using command `bash /provision/setup-docker.sh` +* Deploy using command `bash /provision/run-docker.sh DOCKERNAME APP_PORT SSH_PORT` for example `bash /provision/run-docker.sh staging 8000 8022` +* Profit :) + +## HTTPS Server and Domain Name +* Add your domain name on the `ALLOWED_HOSTS` field on `kape/settings.py` +* Add these server blocks on yout NGINX setting (change `DOMAIN_NAME` and `APP_PORT`) + ``` + server { + listen 80; + listen [::]:80; + server_name your.DOMAIN_NAME.com; + return 301 https://$server_name$request_uri; + } + + server { + listen 443 ssl default_server; + listen [::]:443 ssl default_server; + include snippets/ssl-your.DOMAIN_NAME.sslconfiguration.conf; + server_name your.DOMAIN_NAME.com; + location / { + proxy_pass http://localhost:APP_PORT; + proxy_set_header Host $host; + proxy_set_header X-Real-IP $remote_addr; + proxy_set_header X-Forwarded-Proto $scheme; + } + } + ``` + +## Git push on the production server +* You can push directly to production git if SSH port is not blocked +* In the repo directory, run `git remote add production ssh://kape@DOMAIN_NAME:SSH_PORT/home/kape.git` +* To push to production server, run `git push production master` (default docker ssh password : yukcarikape) + + +# How To Build for Development Environment + +## Requirements + +* NodeJS + Npm Package Manager +* Python + Pip Package Manager +* PostgreSQL + +## Installation + +* `git clone https://gitlab.com/PPL2017csui/PPLA1.git kape` +* `cd kape` +* `npm install` +* `pip install -r requirements.txt` +* `psql` insert these queries: + * `create user kape password 'kape';` to create user. + * `create database kape;` to create database. + * if you want to create another database user, please update the setting file `kape/settings.py` +* `python manage.py migrate` To migrate all the database + +## Run Development Mode App + +* `npm run webpack` to run webpack server. (if you don't want live update run `npm run build`, if you want the uglified+zipped+production ready version run `npm run build-production`) +* on a different terminal window, run `python manage.py runserver` to run API server + +### Now go to localhost:8000 and you'll see the App running! + +## Run Unit Test + +* Frontend test, run `npm run webpack` +* Backend test, run `python manage.py test` -- GitLab From 38e1214b480c97f1339f3fb41904c977e21336cf Mon Sep 17 00:00:00 2001 From: Daya Adianto <dayaadianto@cs.ui.ac.id> Date: Tue, 24 Sep 2019 21:35:45 +0700 Subject: [PATCH 53/53] Fix README --- README.md | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 746c258b..30205460 100755 --- a/README.md +++ b/README.md @@ -7,6 +7,7 @@ - [Install](#install) - [Running Development Mode (Classic)](#running-development-mode-classic) - [Running Development Mode (Containerised)](#running-development-mode-containerised) +- [References](#references) - [License](#license) ## Install @@ -64,7 +65,7 @@ pip install -r requirements.txt If you are cloning the project for the first time into your local development environment, you need to create a new database user named `kape` and create a new database with the same name for this project. Do not forget to set the -ownership of the new database to `kape` user. Once you have finished initalising +ownership of the new database to `kape` user. Once you have finished setting up the database, perform database migration and seeding: ```bash @@ -149,6 +150,11 @@ container. Thus, you can see the app via Web browser by opening > you want to see changes in the codebase reflected in real-time, follow the > classical approach for running the development mode. +## References + +- [The original README file](README.old.md). +- [The original (_upstream_) GitLab project repository](https://gitlab.com/PPL2017csui/PPLA1). + ## License Copyright (c) 2017 PPLA1 Team & Faculty of Computer Science Universitas -- GitLab