PERFORMANCE: repeated bulk queries to records
Description
Typically seen in the log:
01:05:34.039 [cling-36 ] INFO i.t.b.m.i.catalog.finder.RepositoryFinderSupport - query() - interface it.tidalwave.bluemarine2.model.Record
01:05:34.042 [cling-36 ] INFO i.t.b.m.i.catalog.finder.RepositoryFinderSupport - >>>> query: PREFIX foaf: <http://xmlns.com/foaf/0.1/> PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> PREFIX rel: <http://purl.org/vocab/relationship/> PREFIX bm: <http://bluemarine.tidalwave.it/2015/04/mo/> PREFIX mo: <http://purl.org/ontology/mo/> PREFIX vocab: <http://dbtune.org/musicbrainz/resource/vocab/> PREFIX xs: <http://www.w3.org/2001/XMLSchema#> SELECT DISTINCT * WHERE { ?record a mo:Record. ?record rdfs:label ?label. } ORDER BY ?label
01:05:34.044 [cling-36 ] INFO i.t.b.m.i.catalog.finder.RepositoryFinderSupport - >>>> query parameters: []
01:05:34.223 [cling-36 ] INFO i.t.b.m.i.catalog.finder.RepositoryFinderSupport - >>>> query returned 225 entities in 182.389451 msec
01:05:34.587 [cling-36 ] INFO i.t.b.m.i.catalog.finder.RepositoryFinderSupport - query() - interface it.tidalwave.bluemarine2.model.Track
01:05:34.593 [cling-36 ] INFO i.t.b.m.i.catalog.finder.RepositoryFinderSupport - >>>> query: PREFIX foaf: <http://xmlns.com/foaf/0.1/> PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> PREFIX rel: <http://purl.org/vocab/relationship/> PREFIX bm: <http://bluemarine.tidalwave.it/2015/04/mo/> PREFIX mo: <http://purl.org/ontology/mo/> PREFIX vocab: <http://dbtune.org/musicbrainz/resource/vocab/> PREFIX xs: <http://www.w3.org/2001/XMLSchema#> SELECT * WHERE { ?track a mo:Track. ?track rdfs:label ?label. ?track mo:track_number ?track_number. OPTIONAL { ?track bm:diskNumber ?disk_number. } OPTIONAL { ?track bm:diskCount ?disk_count. } ?record a mo:Record. ?record mo:track ?track. ?record rdfs:label ?record_label. ?signal a mo:DigitalSignal. ?signal mo:published_as ?track. ?signal mo:duration ?duration. OPTIONAL { ?signal bm:fileSize ?fileSize. } ?audioFile a mo:AudioFile. ?audioFile mo:encodes ?signal. ?audioFile bm:path ?path. } ORDER BY ?record_label ?disk_number ?track_number ?label
01:05:34.594 [cling-36 ] INFO i.t.b.m.i.catalog.finder.RepositoryFinderSupport - >>>> query parameters: [record, urn:bluemarine:record:7befe4a18b6dd85144ca23c7f064726cdf5a86b6]
01:05:34.716 [cling-36 ] INFO i.t.b.m.i.catalog.finder.RepositoryFinderSupport - >>>> query returned 34 entities in 127.1197 msec
Note that the first query has no parameters. It's likely to be done by PathAwareEntityFinderDelegate.filteredByPath(). In case no obvious solution is available, consider introducing a memory cache.
Environment
Test file(s)
Activity
Cannot be optimized as a native query: the path concept in PathAwareEntity is totally decoupled from the underlying native store. It has to be cached in memory.
Implemented and merged to master. But there are other repeated queries, now clearly not related to PathAwareEntityFinderDelegate. For instance, when browsing new tracks this is seen each time:
00:54:05.531 [cling-5 ] INFO i.t.b.u.m.impl.ContentDirectoryClingAdapter - browse(/music/r+t/r:7a2e6cab72a6b35293c51285c737a0afc1b29088/t:860e870397cc425b8878cdd3d23ded4ee097d971, BrowseMetadata, filter: *, startingIndex: 0, requestedCount: 1, sortCriteria: [])
00:54:05.548 [cling-5 ] INFO i.t.b.m.i.catalog.finder.RepositoryFinderSupport - query() - interface it.tidalwave.bluemarine2.model.Track
00:54:05.557 [cling-5 ] INFO i.t.b.m.i.catalog.finder.RepositoryFinderSupport - >>>> query: PREFIX foaf: <http://xmlns.com/foaf/0.1/> PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> PREFIX rel: <http://purl.org/vocab/relationship/> PREFIX bm: <http://bluemarine.tidalwave.it/2015/04/mo/> PREFIX mo: <http://purl.org/ontology/mo/> PREFIX vocab: <http://dbtune.org/musicbrainz/resource/vocab/> PREFIX xs: <http://www.w3.org/2001/XMLSchema#> SELECT * WHERE { ?track a mo:Track. ?track rdfs:label ?label. ?track mo:track_number ?track_number. OPTIONAL { ?track bm:diskNumber ?disk_number. } OPTIONAL { ?track bm:diskCount ?disk_count. } ?record a mo:Record. ?record mo:track ?track. ?record rdfs:label ?record_label. ?signal a mo:DigitalSignal. ?signal mo:published_as ?track. ?signal mo:duration ?duration. OPTIONAL { ?signal bm:fileSize ?fileSize. } ?audioFile a mo:AudioFile. ?audioFile mo:encodes ?signal. ?audioFile bm:path ?path. } ORDER BY ?record_label ?disk_number ?track_number ?label
00:54:05.559 [cling-5 ] INFO i.t.b.m.i.catalog.finder.RepositoryFinderSupport - >>>> query parameters: [record, urn:bluemarine:record:7a2e6cab72a6b35293c51285c737a0afc1b29088]
00:54:05.710 [cling-5 ] INFO i.t.b.m.i.catalog.finder.RepositoryFinderSupport - >>>> query returned 21 entities in 160.54609 msec
00:54:05.732 [cling-5 ] INFO i.t.b.m.i.catalog.finder.RepositoryFinderSupport - query() - interface it.tidalwave.bluemarine2.model.Record
00:54:05.736 [cling-5 ] INFO i.t.b.m.i.catalog.finder.RepositoryFinderSupport - >>>> query: PREFIX foaf: <http://xmlns.com/foaf/0.1/> PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> PREFIX rel: <http://purl.org/vocab/relationship/> PREFIX bm: <http://bluemarine.tidalwave.it/2015/04/mo/> PREFIX mo: <http://purl.org/ontology/mo/> PREFIX vocab: <http://dbtune.org/musicbrainz/resource/vocab/> PREFIX xs: <http://www.w3.org/2001/XMLSchema#> SELECT DISTINCT * WHERE { ?record a mo:Record. ?record rdfs:label ?label. } ORDER BY ?label
00:54:05.737 [cling-5 ] INFO i.t.b.m.i.catalog.finder.RepositoryFinderSupport - >>>> query parameters: []
00:54:05.926 [cling-5 ] INFO i.t.b.m.i.catalog.finder.RepositoryFinderSupport - >>>> query returned 225 entities in 192.561194 msec
00:54:06.227 [cling-5 ] INFO i.t.b.u.m.impl.ContentDirectoryClingAdapter - >>>> returning BrowseResult(..., 1, 1, 1)
Probably a cache on RepositoryFinders is now undeferrable.
Implemented cache on RepositoryFinderSupport, merged to branch.
But still uncached queries:
01:03:30.027 [cling-11 ] INFO i.t.b.u.m.impl.ContentDirectoryClingAdapter - browse(/music/r+t/r:fe01634ac568a3f7e12b5d2f6e1c0a12c332f4fe/t:7caeae905b3703ab8d96b35135fab6c2a4206a58, BrowseMetadata, filter: *, startingIndex: 0, requestedCount: 1, sortCriteria: [])
01:03:30.029 [cling-11 ] INFO i.t.bluemarine2.model.impl.DefaultCacheManager - >>>> cache miss for ContentDirectoryClingAdapter.BrowseParams(objectId=/music/r+t/r:fe01634ac568a3f7e12b5d2f6e1c0a12c332f4fe/t:7caeae905b3703ab8d96b35135fab6c2a4206a58, browseFlag=BrowseMetadata, filter=*, firstResult=0, maxResults=1, orderby=[])
01:03:30.064 [cling-11 ] INFO i.t.b.m.i.catalog.finder.RepositoryFinderSupport - query() - interface it.tidalwave.bluemarine2.model.Record
01:03:30.068 [cling-11 ] INFO i.t.b.m.i.catalog.finder.RepositoryFinderSupport - >>>> query: PREFIX foaf: <http://xmlns.com/foaf/0.1/> PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> PREFIX rel: <http://purl.org/vocab/relationship/> PREFIX bm: <http://bluemarine.tidalwave.it/2015/04/mo/> PREFIX mo: <http://purl.org/ontology/mo/> PREFIX vocab: <http://dbtune.org/musicbrainz/resource/vocab/> PREFIX xs: <http://www.w3.org/2001/XMLSchema#> SELECT DISTINCT * WHERE { ?record a mo:Record. ?record rdfs:label ?label. } ORDER BY ?label
01:03:30.069 [cling-11 ] INFO i.t.b.m.i.catalog.finder.RepositoryFinderSupport - >>>> query parameters: []
01:03:30.071 [cling-11 ] INFO i.t.b.m.i.catalog.finder.RepositoryFinderSupport - >>>> query returned 225 entities in 5.586848 msec
01:03:30.331 [cling-11 ] INFO i.t.b.m.i.catalog.finder.RepositoryFinderSupport - query() - interface it.tidalwave.bluemarine2.model.Track
01:03:30.339 [cling-11 ] INFO i.t.b.m.i.catalog.finder.RepositoryFinderSupport - >>>> query: PREFIX foaf: <http://xmlns.com/foaf/0.1/> PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> PREFIX rel: <http://purl.org/vocab/relationship/> PREFIX bm: <http://bluemarine.tidalwave.it/2015/04/mo/> PREFIX mo: <http://purl.org/ontology/mo/> PREFIX vocab: <http://dbtune.org/musicbrainz/resource/vocab/> PREFIX xs: <http://www.w3.org/2001/XMLSchema#> SELECT * WHERE { ?track a mo:Track. ?track rdfs:label ?label. ?track mo:track_number ?track_number. OPTIONAL { ?track bm:diskNumber ?disk_number. } OPTIONAL { ?track bm:diskCount ?disk_count. } ?record a mo:Record. ?record mo:track ?track. ?record rdfs:label ?record_label. ?signal a mo:DigitalSignal. ?signal mo:published_as ?track. ?signal mo:duration ?duration. OPTIONAL { ?signal bm:fileSize ?fileSize. } ?audioFile a mo:AudioFile. ?audioFile mo:encodes ?signal. ?audioFile bm:path ?path. } ORDER BY ?record_label ?disk_number ?track_number ?label
01:03:30.340 [cling-11 ] INFO i.t.b.m.i.catalog.finder.RepositoryFinderSupport - >>>> query parameters: [record, urn:bluemarine:record:fe01634ac568a3f7e12b5d2f6e1c0a12c332f4fe]
01:03:30.343 [cling-11 ] INFO i.t.b.m.i.catalog.finder.RepositoryFinderSupport - >>>> query returned 24 entities in 11.076092 msec
01:03:30.369 [cling-11 ] INFO i.t.b.m.i.catalog.finder.RepositoryFinderSupport - query() - interface it.tidalwave.bluemarine2.model.Track
01:03:30.376 [cling-11 ] INFO i.t.b.m.i.catalog.finder.RepositoryFinderSupport - >>>> query: PREFIX foaf: <http://xmlns.com/foaf/0.1/> PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> PREFIX rel: <http://purl.org/vocab/relationship/> PREFIX bm: <http://bluemarine.tidalwave.it/2015/04/mo/> PREFIX mo: <http://purl.org/ontology/mo/> PREFIX vocab: <http://dbtune.org/musicbrainz/resource/vocab/> PREFIX xs: <http://www.w3.org/2001/XMLSchema#> SELECT * WHERE { ?track a mo:Track. ?track rdfs:label ?label. ?track mo:track_number ?track_number. OPTIONAL { ?track bm:diskNumber ?disk_number. } OPTIONAL { ?track bm:diskCount ?disk_count. } ?record a mo:Record. ?record mo:track ?track. ?record rdfs:label ?record_label. ?signal a mo:DigitalSignal. ?signal mo:published_as ?track. ?signal mo:duration ?duration. OPTIONAL { ?signal bm:fileSize ?fileSize. } ?audioFile a mo:AudioFile. ?audioFile mo:encodes ?signal. ?audioFile bm:path ?path. } ORDER BY ?record_label ?disk_number ?track_number ?label
01:03:30.377 [cling-11 ] INFO i.t.b.m.i.catalog.finder.RepositoryFinderSupport - >>>> query parameters: [record, urn:bluemarine:record:fe01634ac568a3f7e12b5d2f6e1c0a12c332f4fe]
01:03:30.379 [cling-11 ] INFO i.t.b.m.i.catalog.finder.RepositoryFinderSupport - >>>> query returned 24 entities in 9.521099 msec
01:03:30.411 [cling-11 ] INFO i.t.b.m.i.catalog.finder.RepositoryFinderSupport - query() - interface it.tidalwave.bluemarine2.model.Record
01:03:30.414 [cling-11 ] INFO i.t.b.m.i.catalog.finder.RepositoryFinderSupport - >>>> query: PREFIX foaf: <http://xmlns.com/foaf/0.1/> PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> PREFIX rel: <http://purl.org/vocab/relationship/> PREFIX bm: <http://bluemarine.tidalwave.it/2015/04/mo/> PREFIX mo: <http://purl.org/ontology/mo/> PREFIX vocab: <http://dbtune.org/musicbrainz/resource/vocab/> PREFIX xs: <http://www.w3.org/2001/XMLSchema#> SELECT DISTINCT * WHERE { ?record a mo:Record. ?record rdfs:label ?label. } ORDER BY ?label
01:03:30.414 [cling-11 ] INFO i.t.b.m.i.catalog.finder.RepositoryFinderSupport - >>>> query parameters: []
01:03:30.416 [cling-11 ] INFO i.t.b.m.i.catalog.finder.RepositoryFinderSupport - >>>> query returned 225 entities in 4.492061 msec
01:03:30.681 [cling-11 ] INFO i.t.b.u.m.impl.ContentDirectoryClingAdapter - >>>> returning BrowseResult(..., 1, 1, 1)
Assignee
Reporter
Labels
id
Operating system(s)
Specs URL
Progress in branch
Components
Fix versions
Affects versions
Priority
