After QC: what still needed a manual check on your last delivery?

[Sean Park]

Hi everyone,
I'm considering building a tool for delivery QC that I might offer for sale in the future. There isn't a product to try or buy yet, and I'd like to understand how people handle this work today.
Thinking about your most recent delivery: after using your usual QC tools, what, if anything, still needed manual investigation or another check on the revised export? How did you handle it? If your existing workflow covered everything, that's useful to hear too.
A brief example here is plenty — please don't share footage, reports, client names or confidential specifications. No files or DMs needed.
Thanks!

 

[Manuel López]

Hi Sean.

In my case there are really two situations.

If I have access to a proper QC tool and it finds an issue, I fix the problem, make a new export and run QC again on the revised master. I would consider it risky to assume that, because the original problem was corrected, the new export no longer needs a full check.

Some errors are already present before export: for example, a reframing operation that accidentally creates a small black border, using a 25 fps clip in a 24 fps timeline (or vice versa), or audio levels that are too high. Those are important, of course, but they usually originate earlier in the workflow.

What concerns me more are errors introduced during the render/export itself, because they may not exist anywhere in the timeline and you have no reason to expect them. I have seen exports with isolated black frames, and also corrupted frames where part of the image was filled with random colours or digital-looking noise.

Those kinds of errors are difficult because, without automated QC, the only reliable way to catch them is to watch the entire master very carefully. On a long programme, that is extremely time-consuming.

The other situation is probably more relevant to me: I don't currently have a dedicated automated QC system that I use regularly. Many of the established solutions are aimed at larger facilities and can be quite expensive.

What I would find useful is a reasonably priced QC application that runs locally on a normal workstation and can reliably flag this kind of unexpected render error, as well as more common delivery issues, while pointing me to anything that still needs visual or manual judgement.

For some specific problems, such as small black borders or repeated-frame / temporal anomalies, I have made my own scripts, but ideally I would prefer those kinds of checks to be part of one practical QC tool.

 

[Dermot Shane]

most of my shows go to tier 1 QC, before i send off, i'll run through Venera PPU to see if there's anything we missed in gradeing, i bypass dead pixels as there's thousands of false positives, making that useless

- typicaly we'll have a two rounds of notes to cure from Qc,
- then the studio / delivery partner does their own Qc, and we have more notes to cure
then i pull delvierables, typicaly:
GAM @ EXR uncompressed / cam orig raster / linearlight
DCDM @ TIFF uncompressed / DCi4k / xyz
CTM @ TIFF uncompressed / UHD / Pq
CTM @ DPX uncompressed / UHD / 709 from Doby
to validate those i use the QC master to check against the exported files,
- run the QC master against the exports with a diffrence key setting IDT to be in the sameish world, and allowing for trim passes, so best if i do that myself, or someone who knows the show well, otherwise it's a stream of minor color diffrences showing up, and me saying: "it'sOK that's as expected" a million times
- run the two files through Drastic Technoligies Video QC compare files, again a ton of false positives due to the diffrent EOTF and trim passes in play

then clone to an external drive with checksums, vaidate and. send off to LTO

places where it would be great to smooth out the process could be before sending to QC, dead pixels in particular, and verifying versions against the QC master, athough both those have so many variables that it's lilkely human eyeballs in play

 

[Evan Anthony]

I will use https://qscan.io/ when needed

 

[Sean Park]

Thanks, Manuel — that's helpful, especially the distinction between timeline issues and errors introduced during export. On the most recent delivery where you encountered one of those export errors, how did you catch it, and roughly how much time did the checking take?

 

[Manuel López]

Thanks, Manuel — that's helpful, especially the distinction between timeline issues and errors introduced during export. On the most recent delivery where you encountered one of those export errors, how did you catch it, and roughly how much time did the checking take?

The most recent case was a DCP. During a private screening with the director, we noticed something odd just before the end credits — a very brief flash. When I checked it afterwards, there was indeed a single black frame that had been introduced during the render.

It wasn't a pure RGB 0,0,0 frame, though. There was a grain layer over it, so every pixel still contained image information. That’s odd. What went wrong was the rendering of the image below. Whether an automated QC system would have flagged it would therefore depend quite a lot on how its black-frame detection was calibrated.

It wasn't something present in the timeline, and there was no reason to expect it. In later exports I actually saw more black frames appear, and the issue was eventually resolved after updating the GPU driver.

So in that case, we caught it simply by watching the finished DCP all the way through. The checking time was therefore essentially the full running time of the film, just to catch a one-frame render error.

That's exactly the kind of problem I'd like an automated QC tool to flag reliably, because these errors are rare but very easy to miss unless someone watches the whole master carefully.

 

[Sean Park]

to validate those i use the QC master to check against the exported files,

Thanks, Dermot — that's a helpful breakdown. On your most recent delivery, could you give one example of a difference between an export and the QC master that needed your judgement, and roughly how long it took to check? No files or client details needed.

 

[Sean Park]

I will use https://qscan.io/ when needed

Thanks, Evan. On the most recent delivery where you used QScan, did it cover the checks you needed, or was there anything you still had to investigate separately afterwards? If it covered everything you needed, that's useful to know too.

 

[Sean Park]

In later exports I actually saw more black frames appear, and the issue was eventually resolved after updating the GPU driver.

Thanks — that makes the example much clearer. On deliveries since the driver update, have you still watched each full export for unexpected render errors, or has your checking routine changed?

 

[Manuel López]

Thanks — that makes the example much clearer. On deliveries since the driver update, have you still watched each full export for unexpected render errors, or has your checking routine changed?

The driver update fixed that particular issue, but I don't assume that means unexpected render errors can never happen again. They are rare, but they can come from different causes.
For an important project, I still try to watch the full final export carefully, simply because the cost of missing a one-frame error can be much greater than the time spent checking it.
On less critical jobs, though, I often just don't have the time to watch every deliverable from beginning to end. That's really the problem for me: I would like an automated first-pass QC that can reliably flag unusual frames or short render glitches, so I can concentrate my time on the suspicious areas rather than having to watch every file in real time — at a price that makes sense for a small operation like mine.

 

[Sean Park]

On less critical jobs, though, I often just don't have the time to watch every deliverable from beginning to end.

That makes sense, Manuel. Have you looked into or tried any existing QC tools for your own setup? If so, which ones, and what kept them from becoming part of your regular workflow?

 

[Manuel López]

That makes sense, Manuel. Have you looked into or tried any existing QC tools for your own setup? If so, which ones, and what kept them from becoming part of your regular workflow?

I don't currently use any dedicated commercial QC software. I've looked at some of the available options, but the main obstacle for me is simply cost.
I work in a small, very price-sensitive market and and I’m sure I don’t make more than about ten deliveries a year, so the cost of the established QC systems is difficult to justify.
For some specific checks I've made a few scripts with the help of ChatGPT, mainly for things like black borders and repeated-frame or temporal anomalies. I use those when they're useful, but otherwise my QC still relies heavily on visual inspection.
So for me, the gap is really an affordable local QC tool that makes sense for a small operation with relatively low delivery volume.

 

[Manuel López]

I should add that this is just my own experience, and probably not the most typical situation among members of this forum.

There are people here working in large facilities, as well as medium and small boutiques, often in much larger markets than mine, so their QC needs, delivery volume and economics may be quite different.

 

[Dermot Shane]

Thanks, Dermot — that's a helpful breakdown. On your most recent delivery, could you give one example of a difference between an export and the QC master that needed your judgement, and roughly how long it took to check? No files or client details needed.

the HDR hero grade that went through Qc was not matching the DCDM grade (as expected, we did a trim pass for 48nit), it took about 3 hours in a conform suite to find all of the area's of concern and then an hour a gradeing suite to validate and sign off

 

[Sean Park]

it took about 3 hours in a conform suite to find all of the area's of concern

Thanks, Dermot — that helps put the checking time in context. What came out of that review: did anything need correcting, or were the differences all expected from the trim pass?

 

[Sean Park]

the main obstacle for me is simply cost.

Thanks, Manuel — I appreciate the caveat about your delivery volume. I found that QScan offers subscription and outright-purchase options, while Venera also offers pay-per-use, so I shouldn't assume the same cost barrier for each. Of the tools you looked at, which came closest to fitting your needs, and what pricing or minimum commitment put it out of reach?

 

[Manuel López]

I haven't actually tried any of the commercial QC tools yet.
QScan One does have a demo, though, so I'm going to give that a proper try when I can. It may be a useful exercise just to see how a dedicated QC application fits into my workflow and what it catches compared with my current manual checks and scripts.
Hopefully the demo gives enough access to understand the workflow properly, and that the documentation/manual is available to trial users as well.

 

[William Watson]

The following have caught us out on the past, (we are factual UK based)
- missing text less elements from tail/missed on the textless version - only caught during crunch time when sold internationally, sometimes years later)
- audio stem incorrect (one line of commentary slipped into a DME stem), and if we're delivering a 12 1 hours in not losing an assistant to listening to 60+ hours of stems
- 5.1 mapping (we do post in-house, except for dub is done externally)
- fact checking from production

 

[Sean Park]

one line of commentary slipped into a DME stem

Thanks, William — that’s a useful example. In that DME stem case, what checks had been done before delivery, and how was the commentary eventually spotted? No files or client details needed.

 

[Sean Park]

QScan One does have a demo, though, so I'm going to give that a proper try when I can.

Thanks, Manuel — that sounds like a useful comparison. If you do try the demo, your impressions of the workflow and what it catches compared with your scripts would be very helpful. No pressure, and I appreciate the clarification.