Skip to content

Conversation

maxhbr
Copy link
Member

@maxhbr maxhbr commented Aug 16, 2018

For the command line usage of monk, one has to provide the license knowledgebase in some way. Usually this is done by connecting the monk application directly to the fossology database and pulling the licenses from there.

For a lightweight setup this is problematic, since one wants to have as little overhead and dependencies as possible.

For that this PR creates the possibility to serialize the knowledgebase using the monk cli tool. The created file can be used to run monk standalone.

standalone usage example / how to test:

(e.g. in the vagrant vm)

$ ./src/monk/agent/monk -s /tmp/monk.knowledgebase
Write knowledgebase to /tmp/monk.knowledgebase
$ sudo /etc/init.d/postgresql stop
 * Stopping PostgreSQL 9.3 database server
$ ./src/monk/agent/monk -k /tmp/monk.knowledgebase LICENSE
found diff match between "LICENSE" and "GPL-2.0" (rf_pk=317); rank 88; diffs: {t[484+18] M0 s[0+18], t[503+139] MR s[19+132], t[643+22] M0 s[152+22], t[666+193] MR s[175+35], t[860+22] M0 s[211+22], t[883+154] MR s[234+661], t[1038+15] M0 s[896+15], t[1054] M- s[912+629], t[1054+1836] M0 s[1542+1831], t[2912+26] M+ s[3377], t[2942+12833] M0 s[3377+12652], t[15781+4] MR s[16033+3], t[15786+35] M0 s[16037+35], t[15822+7] MR s[16073+2], t[15830+15] M0 s[16076+15], t[15846+49] MR s[16092+41], t[15901+703] M0 s[16137+675], t[16605+10] MR s[16814+11], t[16616+1072] M0 s[16826+1068], t[17692+23] MR s[17898+21], t[17716+375] M0 s[17920+372]}
found diff match between "LICENSE" and "LGPL-2.1+" (rf_pk=8); rank 99; diffs: {t[18111+5992] M0 s[0+5923], t[24123+33] M+ s[5927], t[24160+19078] M0 s[5927+18791], t[43244+4] MR s[24722+3], t[43249+35] M0 s[24726+35], t[43285+7] MR s[24762+2], t[43293+15] M0 s[24765+15], t[43309+49] MR s[24781+40], t[43364+1157] M0 s[24825+1122], t[44525+23] MR s[25951+21], t[44549+69] M0 s[25973+68]}

the updated help message:

$ ./src/monk/agent/monk -h
Usage:

As CLI tool using the licenses from the FOSSology database:
       ./src/monk/agent/monk [options] file [file [...]]
               options:
                  -h          :: help (print this message), then exit.
                  -c config   :: specify the directory for the system configuration.
                  -v          :: verbose output.
                  -J          :: JSON output.
                  file        :: scan file and print licenses detected within it.
                  -V          :: print the version info, then exit.

Save knowledgebase to knowledgebaseFile for offline usage without db:
       ./src/monk/agent/monk [options] -s knowledgebaseFile
               options:
                  -c config   :: specify the directory for the system configuration.

Use previously saved knowledgebaseFile for offline usage without db.
       ./src/monk/agent/monk -k knowledgebaseFile [options] file [file [...]]
               options:
                  -J          :: JSON output.
                  file        :: scan file and print licenses detected within it.

The following should only be called by the FOSSology scheduler:
       ./src/monk/agent/monk --scheduler_start [options]
               options:
                  -c config   :: specify the directory for the system configuration.
                  --userID i  :: the id of the user that created the job
                  --groupID i :: the id of the group of the user that created the job
                  --jobID i   :: the id of the job

At a later point one could add support for build time generation of the knowledgebase from the information conained in the source tree.

@ghost ghost assigned maxhbr Aug 16, 2018
@ghost ghost added the needs code review label Aug 16, 2018
@maxhbr maxhbr force-pushed the maxhbr/makeMonkCliIdepententFromDB_2 branch from 958c3a2 to 586bb79 Compare August 16, 2018 13:47
@ghost ghost added the needs code review label Aug 16, 2018
@maxhbr

This comment has been minimized.

@ghost ghost added the needs code review label Aug 16, 2018
@maxhbr maxhbr changed the title feat(monk): allow monk to run standalone WIP: feat(monk): allow monk to run standalone Aug 16, 2018
@ghost ghost added the needs code review label Aug 16, 2018
@ghost ghost added needs code review and removed WIP labels Aug 16, 2018
@ghost ghost added needs code review and removed WIP labels Aug 16, 2018
@maxhbr maxhbr force-pushed the maxhbr/makeMonkCliIdepententFromDB_2 branch from b426d12 to f00b313 Compare August 16, 2018 18:50
@ghost ghost added needs code review and removed WIP labels Aug 16, 2018
@maxhbr

This comment has been minimized.

@maxhbr maxhbr force-pushed the maxhbr/makeMonkCliIdepententFromDB_2 branch from f00b313 to 266d5a7 Compare August 27, 2018 04:56
@ghost ghost added needs code review and removed WIP labels Aug 27, 2018
@maxhbr

This comment has been minimized.

maxhbr added 3 commits August 28, 2018 10:29
CLI and scheduler code is now moved to the appropriate files. This also reworks
and streamlines the command line argument parsing.
This allows monk to run without a connected DB as source of truth
especially for serialization and CLI_OFFLINE
@maxhbr maxhbr force-pushed the maxhbr/makeMonkCliIdepententFromDB_2 branch from 266d5a7 to ee154ea Compare August 28, 2018 08:48
@maxhbr maxhbr changed the title WIP: feat(monk): allow monk to run standalone feat(monk): allow monk to run standalone Aug 28, 2018
@maxhbr maxhbr removed their assignment Aug 28, 2018
@GMishx
Copy link
Member

GMishx commented Aug 29, 2018

Is the PR ready @maxhbr ?

@maxhbr
Copy link
Member Author

maxhbr commented Aug 29, 2018

Yes

Copy link
Member

@GMishx GMishx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes looks good.

Tested, working fine.

@maxhbr
Copy link
Member Author

maxhbr commented Sep 11, 2018

@GMishx merge?

Copy link
Contributor

@ag4ums ag4ums left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

code looks good

@ag4ums ag4ums merged commit 81a78b2 into fossology:master Sep 17, 2018
@ghost ghost removed the needs code review label Sep 17, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants