We're updating the issue view to help you get more done. 

PERFORMANCE: repeated bulk queries to records

Description

Typically seen in the log:

01:05:34.039 [cling-36 ] INFO i.t.b.m.i.catalog.finder.RepositoryFinderSupport - query() - interface it.tidalwave.bluemarine2.model.Record
01:05:34.042 [cling-36 ] INFO i.t.b.m.i.catalog.finder.RepositoryFinderSupport - >>>> query: PREFIX foaf: <http://xmlns.com/foaf/0.1/> PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> PREFIX rel: <http://purl.org/vocab/relationship/> PREFIX bm: <http://bluemarine.tidalwave.it/2015/04/mo/> PREFIX mo: <http://purl.org/ontology/mo/> PREFIX vocab: <http://dbtune.org/musicbrainz/resource/vocab/> PREFIX xs: <http://www.w3.org/2001/XMLSchema#> SELECT DISTINCT * WHERE { ?record a mo:Record. ?record rdfs:label ?label. } ORDER BY ?label
01:05:34.044 [cling-36 ] INFO i.t.b.m.i.catalog.finder.RepositoryFinderSupport - >>>> query parameters: []
01:05:34.223 [cling-36 ] INFO i.t.b.m.i.catalog.finder.RepositoryFinderSupport - >>>> query returned 225 entities in 182.389451 msec
01:05:34.587 [cling-36 ] INFO i.t.b.m.i.catalog.finder.RepositoryFinderSupport - query() - interface it.tidalwave.bluemarine2.model.Track
01:05:34.593 [cling-36 ] INFO i.t.b.m.i.catalog.finder.RepositoryFinderSupport - >>>> query: PREFIX foaf: <http://xmlns.com/foaf/0.1/> PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> PREFIX rel: <http://purl.org/vocab/relationship/> PREFIX bm: <http://bluemarine.tidalwave.it/2015/04/mo/> PREFIX mo: <http://purl.org/ontology/mo/> PREFIX vocab: <http://dbtune.org/musicbrainz/resource/vocab/> PREFIX xs: <http://www.w3.org/2001/XMLSchema#> SELECT * WHERE { ?track a mo:Track. ?track rdfs:label ?label. ?track mo:track_number ?track_number. OPTIONAL { ?track bm:diskNumber ?disk_number. } OPTIONAL { ?track bm:diskCount ?disk_count. } ?record a mo:Record. ?record mo:track ?track. ?record rdfs:label ?record_label. ?signal a mo:DigitalSignal. ?signal mo:published_as ?track. ?signal mo:duration ?duration. OPTIONAL { ?signal bm:fileSize ?fileSize. } ?audioFile a mo:AudioFile. ?audioFile mo:encodes ?signal. ?audioFile bm:path ?path. } ORDER BY ?record_label ?disk_number ?track_number ?label
01:05:34.594 [cling-36 ] INFO i.t.b.m.i.catalog.finder.RepositoryFinderSupport - >>>> query parameters: [record, urn:bluemarine:record:7befe4a18b6dd85144ca23c7f064726cdf5a86b6]
01:05:34.716 [cling-36 ] INFO i.t.b.m.i.catalog.finder.RepositoryFinderSupport - >>>> query returned 34 entities in 127.1197 msec

Note that the first query has no parameters. It's likely to be done by PathAwareEntityFinderDelegate.filteredByPath(). In case no obvious solution is available, consider introducing a memory cache.

Environment

None

Test file(s)

None

Activity

Show:
Fabrizio Giudici
January 10, 2017, 1:15 AM

Cannot be optimized as a native query: the path concept in PathAwareEntity is totally decoupled from the underlying native store. It has to be cached in memory.

Fabrizio Giudici
January 11, 2017, 12:23 AM
Edited

Implemented and merged to master. But there are other repeated queries, now clearly not related to PathAwareEntityFinderDelegate. For instance, when browsing new tracks this is seen each time:

00:54:05.531 [cling-5 ] INFO i.t.b.u.m.impl.ContentDirectoryClingAdapter - browse(/music/r+t/r:7a2e6cab72a6b35293c51285c737a0afc1b29088/t:860e870397cc425b8878cdd3d23ded4ee097d971, BrowseMetadata, filter: *, startingIndex: 0, requestedCount: 1, sortCriteria: [])
00:54:05.548 [cling-5 ] INFO i.t.b.m.i.catalog.finder.RepositoryFinderSupport - query() - interface it.tidalwave.bluemarine2.model.Track
00:54:05.557 [cling-5 ] INFO i.t.b.m.i.catalog.finder.RepositoryFinderSupport - >>>> query: PREFIX foaf: <http://xmlns.com/foaf/0.1/> PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> PREFIX rel: <http://purl.org/vocab/relationship/> PREFIX bm: <http://bluemarine.tidalwave.it/2015/04/mo/> PREFIX mo: <http://purl.org/ontology/mo/> PREFIX vocab: <http://dbtune.org/musicbrainz/resource/vocab/> PREFIX xs: <http://www.w3.org/2001/XMLSchema#> SELECT * WHERE { ?track a mo:Track. ?track rdfs:label ?label. ?track mo:track_number ?track_number. OPTIONAL { ?track bm:diskNumber ?disk_number. } OPTIONAL { ?track bm:diskCount ?disk_count. } ?record a mo:Record. ?record mo:track ?track. ?record rdfs:label ?record_label. ?signal a mo:DigitalSignal. ?signal mo:published_as ?track. ?signal mo:duration ?duration. OPTIONAL { ?signal bm:fileSize ?fileSize. } ?audioFile a mo:AudioFile. ?audioFile mo:encodes ?signal. ?audioFile bm:path ?path. } ORDER BY ?record_label ?disk_number ?track_number ?label
00:54:05.559 [cling-5 ] INFO i.t.b.m.i.catalog.finder.RepositoryFinderSupport - >>>> query parameters: [record, urn:bluemarine:record:7a2e6cab72a6b35293c51285c737a0afc1b29088]
00:54:05.710 [cling-5 ] INFO i.t.b.m.i.catalog.finder.RepositoryFinderSupport - >>>> query returned 21 entities in 160.54609 msec
00:54:05.732 [cling-5 ] INFO i.t.b.m.i.catalog.finder.RepositoryFinderSupport - query() - interface it.tidalwave.bluemarine2.model.Record
00:54:05.736 [cling-5 ] INFO i.t.b.m.i.catalog.finder.RepositoryFinderSupport - >>>> query: PREFIX foaf: <http://xmlns.com/foaf/0.1/> PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> PREFIX rel: <http://purl.org/vocab/relationship/> PREFIX bm: <http://bluemarine.tidalwave.it/2015/04/mo/> PREFIX mo: <http://purl.org/ontology/mo/> PREFIX vocab: <http://dbtune.org/musicbrainz/resource/vocab/> PREFIX xs: <http://www.w3.org/2001/XMLSchema#> SELECT DISTINCT * WHERE { ?record a mo:Record. ?record rdfs:label ?label. } ORDER BY ?label
00:54:05.737 [cling-5 ] INFO i.t.b.m.i.catalog.finder.RepositoryFinderSupport - >>>> query parameters: []
00:54:05.926 [cling-5 ] INFO i.t.b.m.i.catalog.finder.RepositoryFinderSupport - >>>> query returned 225 entities in 192.561194 msec
00:54:06.227 [cling-5 ] INFO i.t.b.u.m.impl.ContentDirectoryClingAdapter - >>>> returning BrowseResult(..., 1, 1, 1)

Fabrizio Giudici
January 11, 2017, 12:55 AM

Probably a cache on RepositoryFinders is now undeferrable.

Fabrizio Giudici
January 12, 2017, 1:05 AM

Implemented cache on RepositoryFinderSupport, merged to branch.

But still uncached queries:

01:03:30.027 [cling-11 ] INFO i.t.b.u.m.impl.ContentDirectoryClingAdapter - browse(/music/r+t/r:fe01634ac568a3f7e12b5d2f6e1c0a12c332f4fe/t:7caeae905b3703ab8d96b35135fab6c2a4206a58, BrowseMetadata, filter: *, startingIndex: 0, requestedCount: 1, sortCriteria: [])
01:03:30.029 [cling-11 ] INFO i.t.bluemarine2.model.impl.DefaultCacheManager - >>>> cache miss for ContentDirectoryClingAdapter.BrowseParams(objectId=/music/r+t/r:fe01634ac568a3f7e12b5d2f6e1c0a12c332f4fe/t:7caeae905b3703ab8d96b35135fab6c2a4206a58, browseFlag=BrowseMetadata, filter=*, firstResult=0, maxResults=1, orderby=[])
01:03:30.064 [cling-11 ] INFO i.t.b.m.i.catalog.finder.RepositoryFinderSupport - query() - interface it.tidalwave.bluemarine2.model.Record
01:03:30.068 [cling-11 ] INFO i.t.b.m.i.catalog.finder.RepositoryFinderSupport - >>>> query: PREFIX foaf: <http://xmlns.com/foaf/0.1/> PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> PREFIX rel: <http://purl.org/vocab/relationship/> PREFIX bm: <http://bluemarine.tidalwave.it/2015/04/mo/> PREFIX mo: <http://purl.org/ontology/mo/> PREFIX vocab: <http://dbtune.org/musicbrainz/resource/vocab/> PREFIX xs: <http://www.w3.org/2001/XMLSchema#> SELECT DISTINCT * WHERE { ?record a mo:Record. ?record rdfs:label ?label. } ORDER BY ?label
01:03:30.069 [cling-11 ] INFO i.t.b.m.i.catalog.finder.RepositoryFinderSupport - >>>> query parameters: []
01:03:30.071 [cling-11 ] INFO i.t.b.m.i.catalog.finder.RepositoryFinderSupport - >>>> query returned 225 entities in 5.586848 msec
01:03:30.331 [cling-11 ] INFO i.t.b.m.i.catalog.finder.RepositoryFinderSupport - query() - interface it.tidalwave.bluemarine2.model.Track
01:03:30.339 [cling-11 ] INFO i.t.b.m.i.catalog.finder.RepositoryFinderSupport - >>>> query: PREFIX foaf: <http://xmlns.com/foaf/0.1/> PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> PREFIX rel: <http://purl.org/vocab/relationship/> PREFIX bm: <http://bluemarine.tidalwave.it/2015/04/mo/> PREFIX mo: <http://purl.org/ontology/mo/> PREFIX vocab: <http://dbtune.org/musicbrainz/resource/vocab/> PREFIX xs: <http://www.w3.org/2001/XMLSchema#> SELECT * WHERE { ?track a mo:Track. ?track rdfs:label ?label. ?track mo:track_number ?track_number. OPTIONAL { ?track bm:diskNumber ?disk_number. } OPTIONAL { ?track bm:diskCount ?disk_count. } ?record a mo:Record. ?record mo:track ?track. ?record rdfs:label ?record_label. ?signal a mo:DigitalSignal. ?signal mo:published_as ?track. ?signal mo:duration ?duration. OPTIONAL { ?signal bm:fileSize ?fileSize. } ?audioFile a mo:AudioFile. ?audioFile mo:encodes ?signal. ?audioFile bm:path ?path. } ORDER BY ?record_label ?disk_number ?track_number ?label
01:03:30.340 [cling-11 ] INFO i.t.b.m.i.catalog.finder.RepositoryFinderSupport - >>>> query parameters: [record, urn:bluemarine:record:fe01634ac568a3f7e12b5d2f6e1c0a12c332f4fe]
01:03:30.343 [cling-11 ] INFO i.t.b.m.i.catalog.finder.RepositoryFinderSupport - >>>> query returned 24 entities in 11.076092 msec
01:03:30.369 [cling-11 ] INFO i.t.b.m.i.catalog.finder.RepositoryFinderSupport - query() - interface it.tidalwave.bluemarine2.model.Track
01:03:30.376 [cling-11 ] INFO i.t.b.m.i.catalog.finder.RepositoryFinderSupport - >>>> query: PREFIX foaf: <http://xmlns.com/foaf/0.1/> PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> PREFIX rel: <http://purl.org/vocab/relationship/> PREFIX bm: <http://bluemarine.tidalwave.it/2015/04/mo/> PREFIX mo: <http://purl.org/ontology/mo/> PREFIX vocab: <http://dbtune.org/musicbrainz/resource/vocab/> PREFIX xs: <http://www.w3.org/2001/XMLSchema#> SELECT * WHERE { ?track a mo:Track. ?track rdfs:label ?label. ?track mo:track_number ?track_number. OPTIONAL { ?track bm:diskNumber ?disk_number. } OPTIONAL { ?track bm:diskCount ?disk_count. } ?record a mo:Record. ?record mo:track ?track. ?record rdfs:label ?record_label. ?signal a mo:DigitalSignal. ?signal mo:published_as ?track. ?signal mo:duration ?duration. OPTIONAL { ?signal bm:fileSize ?fileSize. } ?audioFile a mo:AudioFile. ?audioFile mo:encodes ?signal. ?audioFile bm:path ?path. } ORDER BY ?record_label ?disk_number ?track_number ?label
01:03:30.377 [cling-11 ] INFO i.t.b.m.i.catalog.finder.RepositoryFinderSupport - >>>> query parameters: [record, urn:bluemarine:record:fe01634ac568a3f7e12b5d2f6e1c0a12c332f4fe]
01:03:30.379 [cling-11 ] INFO i.t.b.m.i.catalog.finder.RepositoryFinderSupport - >>>> query returned 24 entities in 9.521099 msec
01:03:30.411 [cling-11 ] INFO i.t.b.m.i.catalog.finder.RepositoryFinderSupport - query() - interface it.tidalwave.bluemarine2.model.Record
01:03:30.414 [cling-11 ] INFO i.t.b.m.i.catalog.finder.RepositoryFinderSupport - >>>> query: PREFIX foaf: <http://xmlns.com/foaf/0.1/> PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> PREFIX rel: <http://purl.org/vocab/relationship/> PREFIX bm: <http://bluemarine.tidalwave.it/2015/04/mo/> PREFIX mo: <http://purl.org/ontology/mo/> PREFIX vocab: <http://dbtune.org/musicbrainz/resource/vocab/> PREFIX xs: <http://www.w3.org/2001/XMLSchema#> SELECT DISTINCT * WHERE { ?record a mo:Record. ?record rdfs:label ?label. } ORDER BY ?label
01:03:30.414 [cling-11 ] INFO i.t.b.m.i.catalog.finder.RepositoryFinderSupport - >>>> query parameters: []
01:03:30.416 [cling-11 ] INFO i.t.b.m.i.catalog.finder.RepositoryFinderSupport - >>>> query returned 225 entities in 4.492061 msec
01:03:30.681 [cling-11 ] INFO i.t.b.u.m.impl.ContentDirectoryClingAdapter - >>>> returning BrowseResult(..., 1, 1, 1)

Assignee

Fabrizio Giudici

Reporter

Fabrizio Giudici

Labels

None

id

None

Operating system(s)

None

Specs URL

None

Progress in branch

None

Components

Fix versions

Affects versions

Priority

Major
Configure