Classpath resources not found using layer-tools

[ferblaca]

Context

We are migrating our applications to using layer-tools but we are finding different behaviors in how applications search for classpath resources through the use of the PathMatchingResourcePatternResolver class.

It seems that the main problem is when using as location pattern an expression starting with wildcards, for example: classpath*:/**/*.avsc.

We have created a simple application that reproduces the case we are discussing.

Steps to reproduce

1. Download application:

git clone https://github.com/ferblaca/DemoPathMatchingResourcePatternResolverApplication.git

2. Compile:

cd DemoPathMatchingResourcePatternResolverApplication
mvn clean install

3. Extract the FATJAR with layer-tools and start the application:

cd boot/target
java -Djarmode=tools -jar boot-0.0.1-SNAPSHOT.jar  extract --layers --destination extracted
cd extracted/application
mkdir lib
cp ../*/lib/*.jar lib/.
java -jar boot-0.0.1-SNAPSHOT.jar

Expected outcome

Using as AntPattern path classpath*:/**/*.avsc the application finds all the .avsc files that are in the classpath inside the jar resources of the artifact module:

+++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++ Resource: URL [jar:file:/home/fbc/workspaces/ferblaca/demoPathMatchingResolver/demoPathMatchingResourcePatternResolver/boot/target/extracted.fer/application/lib/artifact-0.0.1-SNAPSHOT.jar!/demo/statics/avro/file.avsc]
++++++++++ Resource: URL [jar:file:/home/fbc/workspaces/ferblaca/demoPathMatchingResolver/demoPathMatchingResourcePatternResolver/boot/target/extracted.fer/application/lib/artifact-0.0.1-SNAPSHOT.jar!/demo/statics/avro/file3.avsc]
++++++++++ Resource: URL [jar:file:/home/fbc/workspaces/ferblaca/demoPathMatchingResolver/demoPathMatchingResourcePatternResolver/boot/target/extracted.fer/application/lib/artifact-0.0.1-SNAPSHOT.jar!/demo/statics/avro/file2.avsc]
+++++++++++++++++++++++++++++++++++++++++++++++++

Observed outcome

The application running in layer-tools mode does NOT find the .avsc resources:

+++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++ NO RESOURCES FOUND!!!
++++++++++++++++++++++++++++++++++++++++++++++++

Details of tests performed

Application Type	AntPattern Path	Result
FATJAR	classpath*:/**/*.avsc	🆗
LAYER-TOOLS	classpath*:/**/*.avsc	❌
LAYER-TOOLS	classpath*:/demo/**/*.avsc	🆗

Versions

Spring-Boot version 3.3.4

[mhalbritter]

The behavior can be observed without all that copying by leaving out --layers:

cd boot/target
java -Djarmode=tools -jar boot-0.0.1-SNAPSHOT.jar  extract --destination extracted
cd extracted/
java -jar boot-0.0.1-SNAPSHOT.jar

[mhalbritter]

I think this warning applies here (from the JavaDoc of PathMatchingResourcePatternResolver):

WARNING: Note that "classpath*:" when combined with Ant-style patterns will only work reliably with at least one root directory before the pattern starts, unless the actual target files reside in the file system.

[mhalbritter]

For example this:

this.resourceResolver.getResources("classpath:/**/file.avsc")

doesn't work in the Uber jar and in the extracted version.

this.resourceResolver.getResources("classpath:/demo/**/file.avsc")

works in uber jar and in the extracted version.

[ferblaca]

@mhalbritter thank you very much for answering so fast!

We are still wondering why using classpath*:/**/*.avsc does work with UBERJAR and not for EXTRACTED...

Using this.resourceResolver.getResources("classpath*:/**/*.avsc"):

1. With Uberjar results:

+++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++ Resource: URL [jar:nested:/home/fbc/workspaces/ferblaca/demoPathMatchingResolver/demoPathMatchingResourcePatternResolver/boot/target/boot-0.0.1-SNAPSHOT.jar/!BOOT-INF/lib/artifact-0.0.1-SNAPSHOT.jar!/demo/statics/avro/file.avsc]
++++++++++ Resource: URL [jar:nested:/home/fbc/workspaces/ferblaca/demoPathMatchingResolver/demoPathMatchingResourcePatternResolver/boot/target/boot-0.0.1-SNAPSHOT.jar/!BOOT-INF/lib/artifact-0.0.1-SNAPSHOT.jar!/demo/statics/avro/file3.avsc]
++++++++++ Resource: URL [jar:nested:/home/fbc/workspaces/ferblaca/demoPathMatchingResolver/demoPathMatchingResourcePatternResolver/boot/target/boot-0.0.1-SNAPSHOT.jar/!BOOT-INF/lib/artifact-0.0.1-SNAPSHOT.jar!/demo/statics/avro/file2.avsc]
+++++++++++++++++++++++++++++++++++++++++++++++++

2. With extracted results:

+++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++ NO RESOURCES FOUND!!!
++++++++++++++++++++++++++++++++++++++++++++++++

I am returning to the results as I think I did not explain myself correctly initially.

Note that we are using classpath* with asterisk.

Regards,

[mhalbritter]

Maybe someone else from the team knows more about that stuff than I do. But I don't think this is a bug, because it's documented in the JavaDoc warning. But let's see what the rest of the team makes of it.

[wilkinsona]

It's a bit unfortunate that it works with one form of packaging and not another. It would be interesting to know how the uber jar behaves when using the classic loader.

[mhalbritter]

Classic one:

2024-10-11T14:05:39.475+02:00  INFO 111816 --- [demoPathMatchingResourcePatternResolver] [           main] tchingResourcePatternResolverApplication : +++++++++++++++++++++++++++++++++++++++++++++++++
2024-10-11T14:05:39.475+02:00  INFO 111816 --- [demoPathMatchingResourcePatternResolver] [           main] tchingResourcePatternResolverApplication : ++++++++++ Resource: URL [jar:file:/tmp/DemoPathMatchingResourcePatternResolverApplication/boot/boot-0.0.1-SNAPSHOT-classic.jar!/BOOT-INF/lib/artifact-0.0.1-SNAPSHOT.jar!/demo/statics/avro/file.avsc]
2024-10-11T14:05:39.476+02:00  INFO 111816 --- [demoPathMatchingResourcePatternResolver] [           main] tchingResourcePatternResolverApplication : ++++++++++ Resource: URL [jar:file:/tmp/DemoPathMatchingResourcePatternResolverApplication/boot/boot-0.0.1-SNAPSHOT-classic.jar!/BOOT-INF/lib/artifact-0.0.1-SNAPSHOT.jar!/demo/statics/avro/file2.avsc]
2024-10-11T14:05:39.476+02:00  INFO 111816 --- [demoPathMatchingResourcePatternResolver] [           main] tchingResourcePatternResolverApplication : ++++++++++ Resource: URL [jar:file:/tmp/DemoPathMatchingResourcePatternResolverApplication/boot/boot-0.0.1-SNAPSHOT-classic.jar!/BOOT-INF/lib/artifact-0.0.1-SNAPSHOT.jar!/demo/statics/avro/file3.avsc]
2024-10-11T14:05:39.476+02:00  INFO 111816 --- [demoPathMatchingResourcePatternResolver] [           main] tchingResourcePatternResolverApplication : +++++++++++++++++++++++++++++++++++++++++++++++++

Finds the resources, too.

[wilkinsona]

Thanks, @mhalbritter.

This reminds me somewhat of #1744 where running in an IDE behaved one way and running as an uber jar behaved in another way. We decided not to make any changes there as the uber jar's behavior aligned with that of the JDK's jar support.

Here we appear to have an uber jar behaving one way and a standard JDK jar behaving in another way. If we were to align the behavior, it'd have to be by changing the uber jar to somehow stop this from working which, while more consistent, feels like a step backwards.

[philwebb]

Debugging this and I think there's a potential enhancement that could be made to PathMatchingResourcePatternResolver. The addClassPathManifestEntries method has the following comment:

Determine jar file references from {@code Class-Path} manifest entries (which are added to the {@code java.class.path} JVM system property by the system class loader)

This doesn't actually appear to be the case. The result I see in System.getProperty("java.class.path") is just the single root jar. The manifest entries are missing.

Looking at jdk.internal.loader.URLClassPath there's a parseClassPath method that I wonder if we can replicate in Spring so we can find all the root jars.

[philwebb]

I'm working on something to submit to Framework.