From cf2e5c72e08dacaff5003bb234a6bc5053768d1c Mon Sep 17 00:00:00 2001 From: Ben Selwyn-Smith Date: Fri, 30 Jun 2023 13:46:00 +1000 Subject: [PATCH 1/6] docs: add instructions on using repo finder Signed-off-by: Ben Selwyn-Smith --- docs/source/pages/using.rst | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/docs/source/pages/using.rst b/docs/source/pages/using.rst index ff05c7fa9..3b2788ad0 100644 --- a/docs/source/pages/using.rst +++ b/docs/source/pages/using.rst @@ -168,3 +168,18 @@ Thanks to Datalog’s expressive language model, it’s easy to add exception ru requirement. For example, `the Mysql Connector/J `_ dependency in the Micronaut MuShop project does not pass the ``build_service`` check, but can be manually investigated and exempted if trusted. Overall, policies expressed in Datalog can be enforced by Macaron as part of your CI/CD pipeline to detect regressions or unexpected behavior. + +--------------------------- +Analyzing more dependencies +--------------------------- + +In some cases the dependencies that Macaron discovers lack a direct connection to a repository for it to analyze. To improve results in these instances, the Repository Finding feature can be enabled. This feature makes use of a dependency's identifying information and the matching information that can be found within package managing repositories located on the Internet. + +.. note:: The Repository Finding feature currently only works for Java projects via SCM meta data found within artifact POM files. + +Enabling this feature requires modification of the ``defaults.ini`` config file located at: ``/src/config/``. Under the ``repofinder.java`` header, four options of importance exist: ``find_repos``, ``artifact_repositories``, ``repo_pom_paths``, ``find_parents``. These options behave as follows: + +- ``find_repos`` (Values: True or False) - Enables or disables the Repository Finding feature. +- ``artifact_repositories`` (Values: List of URLs) - Determines the remote artifact repositories to attempt to retrieve dependency information from. +- ``repo_pom_paths`` (Values: List of POM tags) - Determines where to search for repository information in the POM files. E.g. scm.url. +- ``find_parents`` (Values: True or False) - When enabled, the Repository Finding feature will also search for repository URLs in parents POM files of the current dependency. From 70340a1f6351c2b3819af3b649a88af10fe66999 Mon Sep 17 00:00:00 2001 From: Ben Selwyn-Smith Date: Tue, 4 Jul 2023 09:44:05 +1000 Subject: [PATCH 2/6] chore: addressed review feedback Signed-off-by: Ben Selwyn-Smith --- docs/source/pages/using.rst | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/docs/source/pages/using.rst b/docs/source/pages/using.rst index 3b2788ad0..b5836523f 100644 --- a/docs/source/pages/using.rst +++ b/docs/source/pages/using.rst @@ -177,9 +177,12 @@ In some cases the dependencies that Macaron discovers lack a direct connection t .. note:: The Repository Finding feature currently only works for Java projects via SCM meta data found within artifact POM files. -Enabling this feature requires modification of the ``defaults.ini`` config file located at: ``/src/config/``. Under the ``repofinder.java`` header, four options of importance exist: ``find_repos``, ``artifact_repositories``, ``repo_pom_paths``, ``find_parents``. These options behave as follows: +This feature is enabled by default. To disable, or configure its behaviour in other ways, a custom ``defaults.ini`` should be passed to Macaron during execution. Under the ``repofinder.java`` header, five options exist: ``find_repos``, ``artifact_repositories``, ``repo_pom_paths``, ``find_parents``, ``artifact_ignore_list``. These options behave as follows: - ``find_repos`` (Values: True or False) - Enables or disables the Repository Finding feature. - ``artifact_repositories`` (Values: List of URLs) - Determines the remote artifact repositories to attempt to retrieve dependency information from. - ``repo_pom_paths`` (Values: List of POM tags) - Determines where to search for repository information in the POM files. E.g. scm.url. - ``find_parents`` (Values: True or False) - When enabled, the Repository Finding feature will also search for repository URLs in parents POM files of the current dependency. +- ``artifact_ignore_list`` (Values: List of GAs) - The Repository Finding feature will skip any artifact in this list. Format is "GroupId":"ArtifactId". E.g. org.apache.maven:maven + +.. note:: Finding repositories requires at least one remote call, adding some additional overhead to an analysis run. From 09ee1050c3119031488eb720244ddde61fd9763a Mon Sep 17 00:00:00 2001 From: Ben Selwyn-Smith Date: Tue, 4 Jul 2023 13:49:08 +1000 Subject: [PATCH 3/6] chore: addressed review feedback Signed-off-by: Ben Selwyn-Smith --- docs/source/pages/using.rst | 38 ++++++++++++++++++------------------- 1 file changed, 19 insertions(+), 19 deletions(-) diff --git a/docs/source/pages/using.rst b/docs/source/pages/using.rst index b5836523f..9e2f3a547 100644 --- a/docs/source/pages/using.rst +++ b/docs/source/pages/using.rst @@ -110,12 +110,30 @@ With the example above, the generated output reports can be seen here: - `micronaut-core.html <../_static/examples/micronaut-projects/micronaut-core/analyze_with_sbom/micronaut-core.html>`__ - `micronaut-core.json <../_static/examples/micronaut-projects/micronaut-core/analyze_with_sbom/micronaut-core.json>`__ +''''''''''''''''''''''''''' +Analyzing more dependencies +''''''''''''''''''''''''''' + +In some cases the dependencies that Macaron discovers lack a direct connection to a repository for it to analyze. To improve results in these instances, the Repository Finding feature can be enabled. This feature makes use of a dependency's identifying information that can be found using the package registries located on the Internet. + +.. note:: The Repository Finding feature currently only works for Java projects via SCM meta data found within artifact POM files. + +This feature is enabled by default. To disable, or configure its behaviour in other ways, a custom ``defaults.ini`` should be passed to Macaron during execution. Under the ``repofinder.java`` header, five options exist: ``find_repos``, ``artifact_repositories``, ``repo_pom_paths``, ``find_parents``, ``artifact_ignore_list``. These options behave as follows: + +- ``find_repos`` (Values: True or False) - Enables or disables the Repository Finding feature. +- ``artifact_repositories`` (Values: List of URLs) - Determines the remote artifact repositories to attempt to retrieve dependency information from. +- ``repo_pom_paths`` (Values: List of POM tags) - Determines where to search for repository information in the POM files. E.g. scm.url. +- ``find_parents`` (Values: True or False) - When enabled, the Repository Finding feature will also search for repository URLs in parents POM files of the current dependency. +- ``artifact_ignore_list`` (Values: List of GAs) - The Repository Finding feature will skip any artifact in this list. Format is "GroupId":"ArtifactId". E.g. org.apache.maven:maven + +.. note:: Finding repositories requires at least one remote call, adding some additional overhead to an analysis run. + ------------------------------------- Analyzing a locally cloned repository ------------------------------------- If you have a local repository that you want to analyze, Macaron also supports running the analysis against a local repository. - +'''''''''''''''''' Assume that the dir tree at the local repository has the following components: .. code-block:: shell @@ -168,21 +186,3 @@ Thanks to Datalog’s expressive language model, it’s easy to add exception ru requirement. For example, `the Mysql Connector/J `_ dependency in the Micronaut MuShop project does not pass the ``build_service`` check, but can be manually investigated and exempted if trusted. Overall, policies expressed in Datalog can be enforced by Macaron as part of your CI/CD pipeline to detect regressions or unexpected behavior. - ---------------------------- -Analyzing more dependencies ---------------------------- - -In some cases the dependencies that Macaron discovers lack a direct connection to a repository for it to analyze. To improve results in these instances, the Repository Finding feature can be enabled. This feature makes use of a dependency's identifying information and the matching information that can be found within package managing repositories located on the Internet. - -.. note:: The Repository Finding feature currently only works for Java projects via SCM meta data found within artifact POM files. - -This feature is enabled by default. To disable, or configure its behaviour in other ways, a custom ``defaults.ini`` should be passed to Macaron during execution. Under the ``repofinder.java`` header, five options exist: ``find_repos``, ``artifact_repositories``, ``repo_pom_paths``, ``find_parents``, ``artifact_ignore_list``. These options behave as follows: - -- ``find_repos`` (Values: True or False) - Enables or disables the Repository Finding feature. -- ``artifact_repositories`` (Values: List of URLs) - Determines the remote artifact repositories to attempt to retrieve dependency information from. -- ``repo_pom_paths`` (Values: List of POM tags) - Determines where to search for repository information in the POM files. E.g. scm.url. -- ``find_parents`` (Values: True or False) - When enabled, the Repository Finding feature will also search for repository URLs in parents POM files of the current dependency. -- ``artifact_ignore_list`` (Values: List of GAs) - The Repository Finding feature will skip any artifact in this list. Format is "GroupId":"ArtifactId". E.g. org.apache.maven:maven - -.. note:: Finding repositories requires at least one remote call, adding some additional overhead to an analysis run. From 0081830c8e42ebaf5d17d1c431c52498457d38b7 Mon Sep 17 00:00:00 2001 From: Ben Selwyn-Smith Date: Tue, 4 Jul 2023 16:04:54 +1000 Subject: [PATCH 4/6] chore: addressed review feedback Signed-off-by: Ben Selwyn-Smith --- docs/source/pages/using.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/pages/using.rst b/docs/source/pages/using.rst index 9e2f3a547..0e2c7209b 100644 --- a/docs/source/pages/using.rst +++ b/docs/source/pages/using.rst @@ -133,7 +133,7 @@ Analyzing a locally cloned repository ------------------------------------- If you have a local repository that you want to analyze, Macaron also supports running the analysis against a local repository. -'''''''''''''''''' + Assume that the dir tree at the local repository has the following components: .. code-block:: shell From 5bc2ee1aa10d7d0486ad42498a7a4488b028311e Mon Sep 17 00:00:00 2001 From: Ben Selwyn-Smith Date: Wed, 5 Jul 2023 09:38:28 +1000 Subject: [PATCH 5/6] chore: addressed review feedback Signed-off-by: Ben Selwyn-Smith --- docs/source/pages/using.rst | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-) diff --git a/docs/source/pages/using.rst b/docs/source/pages/using.rst index 0e2c7209b..a257d513c 100644 --- a/docs/source/pages/using.rst +++ b/docs/source/pages/using.rst @@ -118,7 +118,7 @@ In some cases the dependencies that Macaron discovers lack a direct connection t .. note:: The Repository Finding feature currently only works for Java projects via SCM meta data found within artifact POM files. -This feature is enabled by default. To disable, or configure its behaviour in other ways, a custom ``defaults.ini`` should be passed to Macaron during execution. Under the ``repofinder.java`` header, five options exist: ``find_repos``, ``artifact_repositories``, ``repo_pom_paths``, ``find_parents``, ``artifact_ignore_list``. These options behave as follows: +This feature is enabled by default. To disable, or configure its behaviour in other ways, a custom ``defaults.ini`` should be passed to Macaron during execution (See :ref:`dump-defaults `, the CLI command to dump the default configurations in ``defaults.ini``. After making changes, see :ref:`analyze ` CLI command for the option to pass the modified ``defaults.ini`` file.) Within the configuration file under the ``repofinder.java`` header, five options exist: ``find_repos``, ``artifact_repositories``, ``repo_pom_paths``, ``find_parents``, ``artifact_ignore_list``. These options behave as follows: - ``find_repos`` (Values: True or False) - Enables or disables the Repository Finding feature. - ``artifact_repositories`` (Values: List of URLs) - Determines the remote artifact repositories to attempt to retrieve dependency information from. @@ -128,6 +128,21 @@ This feature is enabled by default. To disable, or configure its behaviour in ot .. note:: Finding repositories requires at least one remote call, adding some additional overhead to an analysis run. +An example configuration file for utilising this feature: + +.. code-block:: ini + +[repofinder.java] +find_repos = True +artifact_repositories = https://repo.maven.apache.org/maven2 +repo_pom_paths = + scm.url + scm.connection + scm.developerConnection +find_parents = True +artifact_ignore_list = + org.apache.maven:maven + ------------------------------------- Analyzing a locally cloned repository ------------------------------------- From 15048b45c27c341f14f0caf9687f0016a06ecfee Mon Sep 17 00:00:00 2001 From: Ben Selwyn-Smith Date: Wed, 5 Jul 2023 09:45:13 +1000 Subject: [PATCH 6/6] chore: addressed review feedback Signed-off-by: Ben Selwyn-Smith --- .../pages/cli_usage/action_dump_defaults.rst | 2 ++ docs/source/pages/using.rst | 26 +++++++++++-------- 2 files changed, 17 insertions(+), 11 deletions(-) diff --git a/docs/source/pages/cli_usage/action_dump_defaults.rst b/docs/source/pages/cli_usage/action_dump_defaults.rst index b439dc91f..2915573e1 100644 --- a/docs/source/pages/cli_usage/action_dump_defaults.rst +++ b/docs/source/pages/cli_usage/action_dump_defaults.rst @@ -1,6 +1,8 @@ .. Copyright (c) 2023 - 2023, Oracle and/or its affiliates. All rights reserved. .. Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl/. +.. _action_dump_defaults: + ============= Dump Defaults ============= diff --git a/docs/source/pages/using.rst b/docs/source/pages/using.rst index a257d513c..04a312bc0 100644 --- a/docs/source/pages/using.rst +++ b/docs/source/pages/using.rst @@ -118,7 +118,11 @@ In some cases the dependencies that Macaron discovers lack a direct connection t .. note:: The Repository Finding feature currently only works for Java projects via SCM meta data found within artifact POM files. -This feature is enabled by default. To disable, or configure its behaviour in other ways, a custom ``defaults.ini`` should be passed to Macaron during execution (See :ref:`dump-defaults `, the CLI command to dump the default configurations in ``defaults.ini``. After making changes, see :ref:`analyze ` CLI command for the option to pass the modified ``defaults.ini`` file.) Within the configuration file under the ``repofinder.java`` header, five options exist: ``find_repos``, ``artifact_repositories``, ``repo_pom_paths``, ``find_parents``, ``artifact_ignore_list``. These options behave as follows: +This feature is enabled by default. To disable, or configure its behaviour in other ways, a custom ``defaults.ini`` should be passed to Macaron during execution. + +See :ref:`dump-defaults `, the CLI command to dump the default configurations in ``defaults.ini``. After making changes, see :ref:`analyze ` CLI command for the option to pass the modified ``defaults.ini`` file. + +Within the configuration file under the ``repofinder.java`` header, five options exist: ``find_repos``, ``artifact_repositories``, ``repo_pom_paths``, ``find_parents``, ``artifact_ignore_list``. These options behave as follows: - ``find_repos`` (Values: True or False) - Enables or disables the Repository Finding feature. - ``artifact_repositories`` (Values: List of URLs) - Determines the remote artifact repositories to attempt to retrieve dependency information from. @@ -132,16 +136,16 @@ An example configuration file for utilising this feature: .. code-block:: ini -[repofinder.java] -find_repos = True -artifact_repositories = https://repo.maven.apache.org/maven2 -repo_pom_paths = - scm.url - scm.connection - scm.developerConnection -find_parents = True -artifact_ignore_list = - org.apache.maven:maven + [repofinder.java] + find_repos = True + artifact_repositories = https://repo.maven.apache.org/maven2 + repo_pom_paths = + scm.url + scm.connection + scm.developerConnection + find_parents = True + artifact_ignore_list = + org.apache.maven:maven ------------------------------------- Analyzing a locally cloned repository