Forums » Feature Requests Search

Time Offset of detection in the audio stream? New Reply

Author Post
Posts: 3
Registered: Feb 07, 2012

It would be nice to have the time offset of the detection in the original audio stream... For example if I send in a random sample from a song, I want to know how many seconds into the song this sample was detected in...

Does this feature exists in the current API...???

Posts: 5
Registered: Feb 16, 2012

+1

Posts: 138
Registered: Feb 07, 2012

topo: edster9: If you are thinking of the song/identify API, then the results returned do not currently indicate at which offset the query was detected in. I agree that it would be great to have this feature, but for the time being development is concentrated on other areas.

Posts: 3
Registered: Feb 07, 2012

This is for after you have detected what song or audio clip you are dealing with... Basically the idea is that after you have detected the song id, now you want to build some sort of a synchronization application... So keep querying once in a while against just one audio clip to know where you are in the timeline... you can keep syncing with the program even if you are dealing with a DVR so you can detect Fast forward and Rewinds...

Also if it could give you a list of possible time indexes every time would be the best... since a song might have common areas that repeat them selves and it will allow the syncing application to decide which one it want to choose...

I am sort of working on this at my spare time but it would be great addition to the API to have this feature... There are a lot TV shows now that offer an App that can sync with the shows timeline using audio finger printing... And I think it would be a major feature enhancement to this library...

Posts: 138
Registered: Feb 07, 2012

I agree that this would be a fantastic feature. Are we talking specifically about the open source Echoprint software distribution, or do you mean the song/identify API more generally? If it is the former, then I have some ideas for how this could be achieved during the matching process when the song is identified, which would return a list of approximate, possible offsets at which the query was found. If you wanted to contribute patches to implement such a feature (if this is indeed what you are offering to do, although I'm just reading between the lines and not entirely sure what you mean), then we would be very happy to accept them into a feature or development branch in the GitHub distribution of Echoprint and we could work on them together. Once they had undergone testing, from there they could make their way into a proper release.

However, it sounds from what you are saying that you want to take an excerpt from a known track and do some sort of time alignment against that, which is a slightly different but related problem. There are several algorithms out there which can already do this. Again, contributions to do this would be welcome for Echoprint, although their exact nature would indicate how appropriate they would be for the core Echoprint distribution vs some other library.

In any case, we will add them to our list of features to consider. Right now we are working primarily on improving Echoprint's matching accuracy and improving the open source software distribution, but this feature would be good to have in future.

Posts: 3
Registered: Feb 07, 2012

Yes definitely only in the context of Echoprint library and not the song/identify API... If I make good enough progress I would certainly share my efforts with the community...

To elaborate a little more and to clarify on what exactly needs to be enhanced, here is an example...

Currently you can use the echoprint-codegen to generate a fingerprint for a given audio... Now the echoprint-server can be instructed to only work with one fingerprint during the detection process and figure out the detection time index... Currently you can use the server local mode to load just one finger print into memory and work with it...

The detection time index is extremely useful for a synchronization type application... (Example if you tune your TV to a given program, then your app can be synced to events that are taking place a that moment...

Posts: 138
Registered: Feb 07, 2012

edster9: That sounds excellent. Would be very interested to hear about how you go with this. Let us know if you have any questions.

I haven't explicitly tested Echoprint Codegen to see how it performs specifically for this sort of time-alignment task, although due to other tests I have performed I strongly suspect that it would need a few modifications to make it work in this scenario. I'm actually looking into similar sorts of adjustments within the context of "big picture" Echoprint development and matching performance, which is ongoing work, so stay tuned.

We would love to see how Echoprint might be used for synchronization with broadcast media if it is possible to do so. Even if this were to happen, realistically it would be more of a long term goal. This request has come up several times before and it is good to see people interested in this use case. Keep in mind that Echoprint is designed only for music, so I wouldn't expect it to work for arbitrary audio signals.

Posts: 1
Registered: Jul 25, 2012

+1, would be a great feature

Posts: 1
Registered: May 07, 2012

+1 from me too - i would like to see time offset too

Posts: 138
Registered: Feb 07, 2012

Thanks for the feedback, xiiiiiiiiii and zoltanbognar :-)

We're still working on a few other things which take precedence, but this feature is still very much on our list of things we intend to implement.

Best,

Andrew

Posts: 2
Registered: Mar 11, 2013

I am very interested in this feature too. I am working on that.

Posts: 1
Registered: Apr 22, 2013

+1, this would be awesome

Posts: 1
Registered: Jul 07, 2013

+1, I also need it

Posts: 2
Registered: Dec 11, 2013

+1! I'm analyzing a radio streaming, so I take 20 seconds and get it analysed by calling /track/upload.

If that time offset is given, I just need to call API again after current song ends.

Posts: 1
Registered: Jul 30, 2014

+1

Have you seen MusicXmatch? They place the lyrics right were the song is, with the right offset, is there really no way for us to know that offset?

(MusicXmatch uses echonest as everybody knows)

I really need it too jeje

Reply to this Thread

You must log in to post a reply.