Subtitle parsing broken in 1.4.0-alpha02+

Version

Media3 main branch

More version details

Subtitle parsing for a given DASH asset (DRM protected) works fine in 1.4.0-alpha01 but in 1.4.0-alpha02 and 1.4.0-beta01 the playback fails with the following exception. Without provding the asset, can you help figure this out? Any info from the DASH manifest that I could provide to help? Again, same asset plays fine and displays subtitles in 1.4.0-alpha01

androidx.media3.exoplayer.ExoPlaybackException: Source error
      at androidx.media3.exoplayer.ExoPlayerImplInternal.handleIoException(ExoPlayerImplInternal.java:741)
      at androidx.media3.exoplayer.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:711)
      at android.os.Handler.dispatchMessage(Handler.java:102)
      at android.os.Looper.loop(Looper.java:223)
      at android.os.HandlerThread.run(HandlerThread.java:67)
  Caused by: androidx.media3.common.ParserException: SubtitleParser failed.{contentIsMalformed=true, dataType=1}
      at androidx.media3.extractor.text.SubtitleExtractor.parseAndWriteToOutput(SubtitleExtractor.java:258)
      at androidx.media3.extractor.text.SubtitleExtractor.read(SubtitleExtractor.java:161)
      at androidx.media3.exoplayer.source.chunk.BundledChunkExtractor.read(BundledChunkExtractor.java:241)
      at androidx.media3.exoplayer.source.chunk.ContainerMediaChunk.load(ContainerMediaChunk.java:132)
      at androidx.media3.exoplayer.upstream.Loader$LoadTask.run(Loader.java:421)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
      at java.lang.Thread.run(Thread.java:923)
  Caused by: java.lang.IllegalArgumentException
      at androidx.media3.common.util.Assertions.checkArgument(Assertions.java:40)
      at androidx.media3.extractor.text.webvtt.WebvttSubtitle.getEventTime(WebvttSubtitle.java:63)
      at androidx.media3.extractor.text.LegacySubtitleUtil.toCuesWithTiming(LegacySubtitleUtil.java:44)
      at androidx.media3.extractor.text.webvtt.WebvttParser.parse(WebvttParser.java:108)
      at androidx.media3.extractor.text.SubtitleParser.parse(SubtitleParser.java:152)
      at androidx.media3.extractor.text.SubtitleExtractor.parseAndWriteToOutput(SubtitleExtractor.java:238)
      at androidx.media3.extractor.text.SubtitleExtractor.read(SubtitleExtractor.java:161) 
      at androidx.media3.exoplayer.source.chunk.BundledChunkExtractor.read(BundledChunkExtractor.java:241) 
      at androidx.media3.exoplayer.source.chunk.ContainerMediaChunk.load(ContainerMediaChunk.java:132) 
      at androidx.media3.exoplayer.upstream.Loader$LoadTask.run(Loader.java:421) 
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167) 
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641) 
      at java.lang.Thread.run(Thread.java:923)

Devices that reproduce the issue

Nvidia Shield TV
Pixel 7 Pro

Devices that do not reproduce the issue

None known

Reproducible in the demo app?

Yes

Reproduction steps

player.trackSelectionParameters =
  player.trackSelectionParameters.buildUpon()
    .setTrackTypeDisabled(TRACK_TYPE_TEXT, !showSubtitles)
    .setPreferredTextLanguage("eng")
    .build()

Expected result

Playback succeeds even when displaying subtitles

Actual result

Receive ERROR_CODE_PARSING_CONTAINER_MALFORMED 3001 during playback when attempting to display subtitles

Media

Private URL. Please let me know if there is some info I can provide from the DASH manifest

Bug Report

You will email the zip file produced by adb bugreport to [email protected] after filing this issue.

1.4.0-alpha02 moved subtitle parsing to happen during extraction by default (instead of during rendering). I wonder if your problem is resolved by moving it back to during rendering? You can do this by calling both MediaSource.Factory.experimentalParseSubtitlesDuringExtraction(false) and TextRenderer.experimentalSetLegacyDecodingEnabled(true) (see also the release notes: https://github.com/androidx/media/releases/tag/1.4.0-alpha02).

Looking at the code, it seems the exception you're seeing is basically an index out of bounds issue:

media/libraries/extractor/src/main/java/androidx/media3/extractor/text/webvtt/WebvttSubtitle.java

Lines 61 to 65 in 023e9d1

    
           public long getEventTime(int index) { 
        
             Assertions.checkArgument(index >= 0); 
        
             Assertions.checkArgument(index < sortedCueTimesUs.length); 
        
             return sortedCueTimesUs[index]; 
        
           }

I think I can see the issue with the logic here - we need to either not return subtitle.getEventTimeCount() from getStartIndex(...) or check that startIndex < subtitle.getEventTimeCount() before calling long firstEventTimeUs = subtitle.getEventTime(startIndex). I will have a go at writing a test case to reproduce the stack trace you're seeing, and see if this change resolves it.

media/libraries/extractor/src/main/java/androidx/media3/extractor/text/LegacySubtitleUtil.java

Lines 40 to 52 in 023e9d1

    
           int startIndex = getStartIndex(subtitle, outputOptions); 
        
           boolean startedInMiddleOfCue = false; 
        
           if (outputOptions.startTimeUs != C.TIME_UNSET) { 
        
             List<Cue> cuesAtStartTime = subtitle.getCues(outputOptions.startTimeUs); 
        
             long firstEventTimeUs = subtitle.getEventTime(startIndex); 
        
             if (!cuesAtStartTime.isEmpty() 
        
                 && startIndex < subtitle.getEventTimeCount() 
        
                 && outputOptions.startTimeUs < firstEventTimeUs) { 
        
               output.accept( 
        
                   new CuesWithTiming( 
        
                       cuesAtStartTime, 
        
                       outputOptions.startTimeUs, 
        
                       firstEventTimeUs - outputOptions.startTimeUs));

I've added a test that creates the same stack trace, and sent a change the fixes the code to not crash in this case - I hope to include this in 1.4.0-rc01. Thanks for raising this, and providing stack trace details so we could work out the issue.

Fix index out of bounds exception when a `Subtitle` is empty

This should be fixed by the commit above.

Fix index out of bounds exception when a `Subtitle` is empty

@icbaker

Hello @icbaker

I'm testing this out in 1.4.0 and still see the same problem. You are correct in that this is the place the IllegalArgumentException is getting thrown:

 public long getEventTime(int index) { 
   Assertions.checkArgument(index >= 0); 
   Assertions.checkArgument(index < sortedCueTimesUs.length);  <-------------- HERE
   return sortedCueTimesUs[index]; 
 }

and it is indeed because the nextEventTimeIndex is C.INDEX_UNSET, which causes the startIndex to be set to the length of the array

   private static int getStartIndex(Subtitle subtitle, OutputOptions outputOptions) {
    if (outputOptions.startTimeUs == C.TIME_UNSET) {
      return 0;
    }
    int nextEventTimeIndex = subtitle.getNextEventTimeIndex(outputOptions.startTimeUs);
    if (nextEventTimeIndex == C.INDEX_UNSET) {
      return subtitle.getEventTimeCount();        <---------- HERE
    }
    if (nextEventTimeIndex > 0
        && subtitle.getEventTime(nextEventTimeIndex - 1) == outputOptions.startTimeUs) {
      nextEventTimeIndex--;
    }
    return nextEventTimeIndex;
  }

I'm working on changing the setup to try the legacy way and will report on the results there

The array of sortedCueTimesUs is like this from our VTT:

[0, 241000, 241000, 1409000, 1409000, 1442000, 1442000, 1475000, 1475000, 1509000, 1509000, 1576000, 1576000, 1642000, 1642000, 1676000, 1676000, 1742000, 1742000, 1776000, 1776000, 1809000, 1809000, 1843000, 1843000, 2009000, 2009000, 2043000, 2043000, 2076000, 2076000, 2109000, 2109000, 2210000, 2210000, 2243000, 2243000, 2276000, 2276000, 2310000, 2310000, 2610000, 2610000, 2677000, 2677000, 2810000, 2810000, 2844000, 2844000, 2877000, 2877000, 3211000, 3211000, 3244000, 3244000, 3277000, 3277000, 3511000, 3511000, 3544000, 3544000, 3578000, 3578000, 3644000, 3644000, 3678000, 3678000, 3711000, 3711000, 4004000]

But the value we are searching for in Util.binarySearchCeil() is 458500275832 so it seems like our third-party VTTs are not in the expected format somehow...

I'd like to point out that ExoPlayer doesn't crash, everything is caught.

It's just that our playback is broken (we get a 3001) when trying to play with subtitles on

Please can you give us a way to play a stream that reproduces this issue? Without that I'm afraid we're just guessing at the possible cause (and it seems that last time we guessed wrong).

Confirmed our subtitles show properly and no playback errors if we use:

MediaSource.Factory.experimentalParseSubtitlesDuringExtraction(false) TextRenderer.experimentalSetLegacyDecodingEnabled(true)

@JonWatson

@JonWatson Are you able to share the DASH manifest URL via email, without the DRM license/token info? That may be enough for us to reproduce the issue (since it only concerns subtitles, which presumably aren't encrypted).

Yes I can do that! An example manifest URL has been sent to [email protected]

Fix `IndexOutOfBoundsException` in `LegacySubtitleUtil`

Thanks for the test stream, it's been really helpful to look into this issue. I've fixed the IndexOutOfBoundsException with the commit linked above, which solves the playback error, but subtitles are still not shown on your stream.

Looking in your DASH manifest it seems that your subtitles are raw WebVTT (not encapsulated in ISOBMFF / MP4) but are segmented. I believe this combination is not supported by the DASH-IF IOP 5.2.10 (emphasis mine):

Some services store text adaptation sets in stand-alone IMSC1 or WebVTT files, without segmentation or [ISOBMFF] encapsulation.
This document requires:

Timecodes in stand-alone text files SHALL be relative to the period start point.

@presentationTimeOffset SHALL NOT be present and SHALL be ignored by clients if present.

I appreciate that this stream works on 1.3.1, and not on 1.4.0, so even if this content isn't fully spec compliant it still seems like a regression in the library.

I'm still working on plumbing the right timestamps around to get these subtitles showing up again - I haven't quite got it working yet.

@JonWatson

@JonWatson Ah I think the stream stopped working today - would you be able to email a new link so I can continue debugging? Thanks!

Done! Thank you

Pass missing `length` into `SubtitleParser` from `SubtitleExtractor`

@JonWatson

@JonWatson I've made some hacky local changes which get subtitles showing for your stream, but I need to clean them up before being able to submit (because at the moment they will break other use-cases). Unfortunately I think the stream stopped working again today - would you be able to send a new one?

@icbaker

Thanks @icbaker , new manifest has been emailed

Fix `IndexOutOfBoundsException` in `LegacySubtitleUtil`

Pass missing `length` into `SubtitleParser` from `SubtitleExtractor`

enable legacy decoding of text captions

Feat/migrate media3 (#35)

Navigation Menu

Repository navigation

Subtitle parsing broken in 1.4.0-alpha02+ #1516

Version

More version details

Devices that reproduce the issue

Devices that do not reproduce the issue

Reproducible in the demo app?

Reproduction steps

Expected result

Actual result

Media

Bug Report

5 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	public long getEventTime(int index) {
	Assertions.checkArgument(index >= 0);
	Assertions.checkArgument(index < sortedCueTimesUs.length);
	return sortedCueTimesUs[index];
	}

	int startIndex = getStartIndex(subtitle, outputOptions);
	boolean startedInMiddleOfCue = false;
	if (outputOptions.startTimeUs != C.TIME_UNSET) {
	List<Cue> cuesAtStartTime = subtitle.getCues(outputOptions.startTimeUs);
	long firstEventTimeUs = subtitle.getEventTime(startIndex);
	if (!cuesAtStartTime.isEmpty()
	&& startIndex < subtitle.getEventTimeCount()
	&& outputOptions.startTimeUs < firstEventTimeUs) {
	output.accept(
	new CuesWithTiming(
	cuesAtStartTime,
	outputOptions.startTimeUs,
	firstEventTimeUs - outputOptions.startTimeUs));

Subtitle parsing broken in 1.4.0-alpha02+ #1516

Description

Version

More version details

Devices that reproduce the issue

Devices that do not reproduce the issue

Reproducible in the demo app?

Reproduction steps

Expected result

Actual result

Media

Bug Report

Activity

icbaker commented on Jul 1, 2024

icbaker commented on Jul 1, 2024

icbaker commented on Jul 2, 2024

JonWatson commented on Jul 29, 2024

JonWatson commented on Jul 29, 2024

JonWatson commented on Jul 29, 2024

icbaker commented on Jul 29, 2024

5 remaining items

JonWatson commented on Jul 29, 2024

icbaker commented on Jul 31, 2024

JonWatson commented on Jul 31, 2024

icbaker commented on Aug 8, 2024

icbaker commented on Aug 8, 2024

JonWatson commented on Aug 8, 2024

icbaker commented on Aug 16, 2024

JonWatson commented on Aug 16, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions