All the development process is happening inside GitLab server https://git.bgerp.org There we do branches for perform changes, run automatic tests and publish updates.
To the external GitHub https://github.com/Pingvin235/bgerp are published only state of master branch after change’s acceptance. The repository is enough for exploration purposes and Custom development.
Each change is performed in a separated GIT branch. For development of the project we use Standard GIT Workflow. Similar one is used, for example, for GNU/Linux development.
The main idea is that merging is only happening 'downstream', i.e. from the main branch to secondary branches, in order to get to the actual state. Main branch has linear structure as all changes are incorporated via 'patches'. Secondary branches can be then deleted, as all the aggregated information from secondary branches will be present in the main branch.
master is a main project branch and is used for builds.
Each change should be related to a BGERP process with PROCESS_ID, used for all information exchange for a given change.
For each change create a separate GIT branch based on master. Branch’s name should start from p<PROCESS_ID>, then use "-" as delimiters. Example: p11788-link-filter-title
When developing, you can use any commit strategy within a branch: intermediate commits, reverts and resets. We would recommend to push intermediate commits, using GIT repository as a backup copy.
Main branch can be periodically merged to the change branch in order to synchronize with the actual state.
Branch has to add build/change.<PROCESS_ID>.txt file, create it by running
./gradlew touchChanges. This file has to describe new functions, fixes or other changes - each item on a separate row. Format is identical to changes.txt, which accumulates all build/change.<PROCESS_ID>.txt information when release update gets published.
All the necessary documentation changes are done in the branch after the code changes.
Request branch acceptance to master once development and testing completes for a given change by moving BGERP process into Acceptance status.
Process gets closed after check/acceptance, and change gets merged to master as a single commit with a comment starting with the change ID.
|Be sure that your local GIT repo is properly configured.|
All the following GIT commands have to be executed in the project’s directory.
git checkout master && git pull origin master && git checkout -b p12345-my-change-branch
git checkout p12345-my-change-branch && git add . && git commit -am "My changes" && git push
At the first push for a branch you will be asked to set an upstream branch using the command.
git push --set-upstream origin p12345-my-change-branch
For showing the current state of GIT working directory use:
Displays text-rendered GIT graph:
git log --oneline --decorate --all --graph
git checkout p12345-my-change-branch && git pull --rebase && git fetch origin master:master && git merge master
Clean all links to non-existent/deleted branches.
git remote prune origin
Clean GIT repo from unreferenced objects after.
git reflog expire --expire-unreachable=now --all && git gc --prune=now
Clean unneeded GIT LFS files.
git lfs prune
Error: Encountered 7 file(s) that should have been pointers, but weren’t
git rm --cached -r . && git reset --hard
When change is being accepted, 'change' branch is compared with the master. So merge the latest state of master to the 'change' branch before passing a change to acceptance.
GIT commands to be used for merging change from pXXXXX-short-change-description branch to master:
git checkout pXXXXX-short-change-description && git pull git commit --allow-empty -m "MERGED" && git push git checkout master && git pull git merge --squash pXXXXX-short-change-description
Commit with current GIT user.
git commit -am "pXXXXX Some change description."
Or for preserving the author in GitHub. Mapping internal to external mails is available in file
git commit -am "pXXXXX Some change description." --author="Developer Name <firstname.lastname@example.org>"
And finally, push.
|Perform a separate GIT Push after each acceptance in order to correctly publish in the open repository.|
After each acceptance to the master branch CI performs test-and-publish-update-master and test-integration-master jobs, which together prepare data for running Demo System.
Long-lived branches only get changes which are not altering product functionality, for example: localization and documentation fixes, code formatting, tests. BGERP process for a change does not get closed , and acceptance is performed multiple times upon completion of specific change(s).
For building of the manual used Open Source tool PzdcDoc. The source AsciiDoctor and resource files are placed in
srcx/doc directory. The examples of documentation’s format may be found here.
Plugin documentation should have the following structure:
About - common information about;
Setup - how to configure;
Usage - using instructions with screenshots;
Development - info for developers.
Images like screenshots in documentation must be stored in PNG format with .png extension. This extension is configured to be stored in GIT LFS. Use width attribute for limiting size rather than resizing that can sometimes increase a file size.
Documentation changes are recommended to be done at the end of branch development, using changes file as preliminary notes. CI job test-and-publish-update builds documentation automatically for each commit or it might be built locally.
Due the strict references and snippets checking, it is quite possible to have broken state of documentation even without changes in
.adoc files. Something like the following:
2020-05-25 12:17:39,149 INFO DocGenerator [main] Processing: srcx/doc/project.adoc 2020-05-25 12:17:39,844 ERROR Snippet [main] Snippet '../../src/ru/bgcrm/struts/action/MessageAction.java' doesn't start from: 'message.se', line number: 205, content: newProcess.setDescription(message.getSubject()); 2020-05-25 12:17:39,855 ERROR Snippet [main] Snippet '../../src/ru/bgcrm/struts/action/MessageAction.java' doesn't end on: ');', line number: 71, content: if (message == null) 2020-05-25 12:17:39,859 ERROR Snippet [main] Snippet '../../src/ru/bgcrm/struts/action/MessageAction.java' doesn't start from: 'pu', line number: 241, content: 2020-05-25 12:17:39,860 ERROR Snippet [main] Snippet '../../src/ru/bgcrm/struts/action/MessageAction.java' doesn't end on: '}', line number: 253, content: 2020-05-25 12:17:39,911 ERROR Snippet [main] Snippet '../../webapps/WEB-INF/jspf/user/search/search.jsp' doesn't start from: '<div', line number: 1, content: <%@ page contentType="text/html; charset=UTF-8"%> 2020-05-25 12:17:39,911 ERROR Snippet [main] Snippet '../../webapps/WEB-INF/jspf/user/search/search.jsp' doesn't end on: '/div>', line number: 134, content: <%@ include file="/WEB-INF/jspf/shell_title.jsp"%> 2
For such cases here is the fixing algorithm. First, find the failing line in
After that, using branch comparison, find the new rows and change them in the
If documentation was already corrected in the current branch, you can create a mock branch on the last working state.
Use line numbers for searching over failing snippets.
Place human readable changes description with screens (possible link images from a main article) in 00000 changes file.
| You can postpone the complete work and only store screenshots to the
Publication of the product manual runs automatically from long-lived branch p11862-documentation. In the same branch may be made changes for documentation only of the latest release that also will be published.
.gitlab-ci.yml file has configuration for running certain jobs automatically upon each GIT commit. Different jobs are executed within different GIT branches.
Please find below description of CI jobs.
The job is executed in every change branch.
Launches Unit tests and documentation build and validity check. If everything is went without error then publishing update with documentation from the branch to https://bgerp.org/update/<PROCESS_ID>;
|All the published change updates have the same version, equal to the next release.|
The job is executed in master branch.
Does the same as test-and-publish-update but for master branch. As there is no real PROCESS_ID available, it is taken as 00000 published update. The artificial change 00000 can be used for updating to the latest state of master. To this state is periodically reset Demo System.
Runs integration test with DB in container.
The job is executed in master branch.
Runs integration test. Upon successful tests execution, written in DB dump gets extracted for Demo System.
The job is executed in master branch.
Publishes actual source code from
master into an open repository https://github.com/pingvin235/bgerp
The job is executed in long-lived branch p11862-documentation.
The branch contains the documentation sources for the latest release. Launches documentation build and validity check, if no errors are found, publishes on https://bgerp.org/doc/3.0/manual.
Runners are responsible for executing jobs. The project already has some runners available, but you can register additional ones to speed up process.
The list of runners is available here: https://git.bgerp.org/admin/runners There also can be taken registration token.
A runner can be also added are here: https://git.bgerp.org/bgerp/bgerp/-/settings/ci_cd under Runners section.
That can be done on every system with installed Docker.
# delete existing runner if exists docker pull gitlab/gitlab-runner:latest && docker stop gitlab-runner && docker rm gitlab-runner # create and start runner container docker run -d --name gitlab-runner --restart always -v /srv/gitlab-runner/config:/etc/gitlab-runner -v /var/run/docker.sock:/var/run/docker.sock gitlab/gitlab-runner:latest # register runner, use executor: 'docker', URL: https://git.bgerp.org/ docker run --rm -it -v /srv/gitlab-runner/config:/etc/gitlab-runner gitlab/gitlab-runner register
Project is configured in Gradle format (configuration file: build.gradle)
For building and publishing, apart from Java you will need console environment with available scp, ssh and rsync commands.
| Here and below all commands are shown for WSL environment, *NIX will not need
bash -c "./gradlew clean doc"
Resulting HTML files will be present in target/doc. Internal link validation is performed automatically.
|This task is automatically run by CI.|
Build and publish on https://bgerp.org/update update package with a change.
All the updates packages are copies to Web directory:
https://bgerp.org/update/PROCESS_ID The changes file has also copied, and all documentation links there starting from
https://bgerp.org/doc are automatically replaced to the
Users have a capability to update to the change using a PROCESS_ID as an identifier.
Public SSH key or the developer might be added for email@example.com and firstname.lastname@example.org. Before you publish a change update, make sure that
Build is performed from a master branch and can include many accepted changes.
|Be sure that Integration tests are successfully done on master to provide data for Demo System.|
|Stop all running IDEs with opened project directory.|
Perform the following:
./gradlew clean resetProperties updateLibProperties buildUpdateLib updateProperties buildUpdate buildDist
| buildUpdateLib and updateLibProperties tasks check existence of file
Build a Docker Image:
bash build/docker/files.sh && docker build build/docker -t bgerp/bgerp && docker push bgerp/bgerp
Then apply file changes and commit them:
./gradlew changesFile changesRss changesDoc publishCommit
Copy the built artifacts to the server and check the results on https://bgerp.org
bash -c "./gradlew publishRelease"
Check the release commit and make:
Merge the latest state of the master on documentation branch:
git checkout p11862-documentation && git pull --rebase && git merge master && git push
Pick documentation changes back to master.
git checkout master && git pull --rebase && git merge --squash p11862-documentation && git commit -am "p11862 Documentation." && git push
Add news to Web sites about release with a link to the release changes.
|In case of bugfix releases already existing changes document may be renamed, now news is needed.|
When pushing image from a system first you would need to perform login:
docker login --username bgerp
And input access token.
bgerp/bgerp image is based on bgerp/base, which should be rebuilt in case of updating MySQL or Java versions:
bash build/docker/base/files.sh && docker build build/docker/base -t bgerp/base && docker push bgerp/base
JUnit framework is used (
srcx/test directory) Used to test specific algorithms, test do not depend on each other and do not work with DB.
Test are launched locally using the following command:
bash -c "./gradlew clean test"
TestNG framework is used (
Integration test performs initialization an empty DB and filling after it with configuration. Tests form a dependencies graph which defined order and execution parallelism.
By default the tests uses MySQL instance with credentials taken from
bgerp.properties file. For local run use Docker DB Instance for that.
Additionally to the recommended MySQL configuration, check and set in
That will significantly increase table creation speed. After the first successful run, the structure of a DB will remain persistent and re-created faster by this way.
Creation of dump for Windows:
echo DROP DATABASE IF EXISTS bgerp; > ./dump.sql && echo CREATE DATABASE bgerp DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci; >> ./dump.sql && echo USE bgerp; >> ./dump.sql && mysqldump -uDB_USER -pDB_PSWD bgerp --add-drop-database --no-data >> ./dump.sql && type build\bgerp\db_init_end.sql >> ./dump.sql
echo "DROP DATABASE IF EXISTS bgerp;" > ./dump.sql && echo "CREATE DATABASE bgerp DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;" >> ./dump.sql && echo "USE bgerp;" >> ./dump.sql && mysqldump -uDB_USER -pDB_PSWD bgerp --add-drop-database --no-data >> ./dump.sql && cat build/bgerp/db_init_end.sql >> ./dump.sql
Running the tests after:
mysql -uDB_USER -pDB_PSWD < ./dump.sql && gradlew integrationTest -Pdb.user=DB_USER -Pdb.pswd=DB_PSWD -Pskip.dbReset=true