Color.io Has Been Released

Hi Jonathan,
I did have a look and played around a bit when this thread was posted, but I couldn’t find how to set the input colorspace and gamma. Is that even possible?
Thanks
You can set the ACES IDT at the top right (camera icon). The ODTs are under the display icon next to it. You can also override color management for exporting LUTs/DCTLs for third party workflows in the export panel (thanks to @Rune Felix Holm)

User Guide Overview: https://www.color.io/user-guide/core-concepts
User Guide Tools: https://www.color.io/user-guide/color-grading-tools
 
I think what you mentioned would help your potential customers if it were included in your marketing copy. Users are interested in unique, new tools, but they need to understand how they were derived.
OP started this thread on the same day I published the app and marketing site and it was definitely incomplete. I've added a dozen landing pages, user guides and extensive FAQs that go into more detail by now.

On your site you explain what Color.io does using psuedo-technical terms, when clearly given what you said above, you could actually talk about it in an astute manor that would resonate with professionals.
That's a really good point and something I've been thinking about and trying to get right for years. It's super hard to find the right balance here because I'm going through the entire process of having a high level idea for a product to very low-level abstract implementation details, software architecture and algorithms as a single person. Pseudo-technical marketing is probably the result of striking an equal balance between these facets of my work, or, to put it more technical, the linear interpolation between "tools that treat pixels like crystals in an emulsion for painterly, film-like colors" and sth like "sqrt(pow(dot(rgb, vec3(0.59597799, -0.27417610, -0.32180189)), 1.4398876) + pow(dot(rgb, crm_mat_3), 2.2)) * 2.2".

This is really good food for thought though. There's a difference between performing magic for a general audience and selling magic to magicians
 
I think lawyers were involved, so I suspect it was not entirely Juan's decision. But he proved his point.


It can be done with hard work. Have you taken a look at MonoNodes or Demisty-Color? MixingLight also has some terrific tutorials that detail how various looks can be created. And Lowepost and TACResolveTraining has covered different kinds of theories and approaches on how to come up with new (or different) looks.

For me, every time I have to create a new look from scratch for a project, once we're wrapped, I'll save the key part of the grade to a "Look" PowerGrade library, and maybe I can reuse it someday. Maybe not. So much depends on exposure / art direction / lenses / lighting and so on, the same look may fall apart in different situations... but that would be the same thing with a LUT.

Um, I've been doing this for something like seven years. I've also watched some of those Mixing Light tutorials. They are decent, but ultimately still broken in my opinion. Take the color yellow, for instance. With HSV saturation yellow gets turned into orange. Not cool. That is not a problem with Color.io.
 
You can set the ACES IDT at the top right (camera icon). The ODTs are under the display icon next to it. You can also override color management for exporting LUTs/DCTLs for third party workflows in the export panel (thanks to @Rune Felix Holm)

User Guide Overview: https://www.color.io/user-guide/core-concepts
User Guide Tools: https://www.color.io/user-guide/color-grading-tools

You mention the ability to export DCTLs, but so far I have only found the option to export LUTs. Will this be an added feature later?
 
You can set the ACES IDT at the top right (camera icon). The ODTs are under the display icon next to it. You can also override color management for exporting LUTs/DCTLs for third party workflows in the export panel (thanks to @Rune Felix Holm)

User Guide Overview: https://www.color.io/user-guide/core-concepts
User Guide Tools: https://www.color.io/user-guide/color-grading-tools

Thanks, I somehow totally missed that button when I first gave it a try.

I've been playing with it, it's pretty cool what you build!

Just one last question, the part I couldn't see without account... What kind of LUTs does it export?I understand .cube and the size etc, but are they supposted to be used in ACES in Resolve (let's take Resolve as example)? What colorspace and eotf are the LUTs created for? Same question kind of applies to DCTL export. How would you use that on, let's say, an Arri LogC image in Resolve. And what's the difference between exporting a LUT vs exporting as DCTL? Sorry, that was 3 questions!
 
You mention the ability to export DCTLs, but so far I have only found the option to export LUTs. Will this be an added feature later?

DCTL export is at the very bottom of the LUT Formats List.
Export -> 3D LUT -> Destination -> DCTL (Beta)

It's not offering much over LUTs at this point tbh, just a little UI in Resolve to bypass and separately set intensity for chroma and luma.
I'm planning to port more of the core functionality into DCTLs so users can design a base look in Color.io, export as a DCTL and then fine tune individual parameters inside Resolve to make shot-specific adjustments. What would your main use case for DCTLs be over using LUTs?
 
Just one last question, the part I couldn't see without account... What kind of LUTs does it export?I understand .cube and the size etc, but are they supposted to be used in ACES in Resolve (let's take Resolve as example)? What colorspace and eotf are the LUTs created for? Same question kind of applies to DCTL export. How would you use that on, let's say, an Arri LogC image in Resolve. And what's the difference between exporting a LUT vs exporting as DCTL? Sorry, that was 3 questions!

These are the LUT formats currently supported:
Screenshot%202023-03-01%20at%2021.08.03.png


And then there's also the option to override the color management for export so that you can embed the look you've created in Color.io into an ACES working space project or timeline. This will create a LUT that expects the input and output to be whatever color space you have selected. So if you export for ACEScct, you would need a node with a LOG-C to ACEScct IDT, followed by the Color.io LUT, followed by a CST or ODT that goes from ACEScct to your desired output. The same is true for DCTLs but please see my reply to @Aaron Kessler above for gotchas.
Screenshot%202023-03-01%20at%2021.08.23.png
 
DCTL export is at the very bottom of the LUT Formats List.
Export -> 3D LUT -> Destination -> DCTL (Beta)

It's not offering much over LUTs at this point tbh, just a little UI in Resolve to bypass and separately set intensity for chroma and luma.
I'm planning to port more of the core functionality into DCTLs so users can design a base look in Color.io, export as a DCTL and then fine tune individual parameters inside Resolve to make shot-specific adjustments. What would your main use case for DCTLs be over using LUTs?

My main use for DCTLs inside resolve would be 1) that I can make adjustments while referring to scopes, and 2) that I could see my adjustments realtime. Currently, I find that the look does not translate 100% in Resolve and I have to do some guesswork in Color.io as to what the actual Resolve result will be. It's also possible I could be doing something wrong...
 
I find that the look does not translate 100% in Resolve and I have to do some guesswork in Color.io as to what the actual Resolve result will be.
Hmm that doesn't seem right... the point of color managed workflows is to ensure absolute consistency and no guesswork.

A few things to note:
Color.io uses ACES version 1.3 with reference gamut compression enabled.
When you export a LUT or DCTL to Resolve, you likely want to export only the "look" part of the LUT, without any IDT or ODT baked into it.
You can bypass the IDT and ODT by selecting your preferred ACES working space at the bottom of the export panel in Color.io:

Screenshot%202023-03-01%20at%2021.08.23.png


To get a 1:1 match between Color.io and Resolve, you need to select ACES 1.3, use the same IDT (with reference gamut compression enabled) and ODT.
If, for example, your working with Alexa Footage in ACEScct, on an sRGB monitor, your node setup would look like this:

1. ACES Transform :: Input Transform: Alexa Log-C, Output Transform: ACEScct
Screenshot%202023-03-02%20at%2000.55.12.png


2. Color.io LUT that was exported for ACEScct working space

3. ACES Transform :: Input Transform: ACEScct, Output Transform: sRGB
Screenshot%202023-03-02%20at%2000.55.40.png
 
Last edited by a moderator:
What does "float" mean in colorist terminology? Below 0.0 and above 1.0?

Pepo is referring to integer vs floating point math. Integer math being RGB values on a fixed number scale based on bit depth (10-bit: 0-1023), and floating point math being any fractional value from 0 to 1 expressed as a floating point number. "Floating point" refers to the fact that the decimal point can move to accommodate different values.

Pepo's "question" is correct so to speak. LUTs are processed using integer values, where DCTLs are actually strings of code used to program shaders that are calculated on a GPU using 32-bit floating point math. Color calculations done in integer math are subject to clipping if you push RGB values beyond the available scale. The floating point math used in DCTLs is protected from that because of it's ability to represent a much larger range of values. This is really helpful in color processing applications, especially ones that use a node-based processing tree like Resolve or Fusion.
 
Pepo is referring to integer vs floating point math. Integer math being RGB values on a fixed number scale based on bit depth (10-bit: 0-1023), and floating point math being any fractional value from 0 to 1 expressed as a floating point number. "Floating point" refers to the fact that the decimal point can move to accommodate different values.

Pepo's "question" is correct so to speak. LUTs are processed using integer values, where DCTLs are actually strings of code used to program shaders that are calculated on a GPU using 32-bit floating point math. Color calculations done in integer math are subject to clipping if you push RGB values beyond the available scale. The floating point math used in DCTLs is protected from that because of it's ability to represent a much larger range of values. This is really helpful in color processing applications, especially ones that use a node-based processing tree like Resolve or Fusion.

So just the normal definition of float vs integer, gotcha. In that case there would be no difference because the color engine as well as the LUTs exported from Color.io have up to 64-bit floating point precision. There are some LUT formats for in-camera use that are encoded as 10/12-bit integers but that's not how most LUTs for post production work in my experience.

I think in theory a case could be made that algorithmic transformations offer greater color transform stability because of the continuous per-pixel evaluation of the shader math as opposed to "filling in the gaps" in LUTs using interpolation. In practice however, you won't be able to see any difference with homogenous float LUTs with a patch density of 33x33x33 and higher when using tetrahedral interpolation.
 
LUTs are processed using integer values
I don't think that is correct. Resolve certainly doesn't process LUTs using integer values. Most if not all applications work with floats under the hood and even if you were to load an integer based lut into one of these float enabled apps, the LUT values would be piped through an integer to float conversion step, likely with intermediate dithering applied to fake some precision.
 
I don't think that is correct. Resolve certainly doesn't process LUTs using integer values.

Classically, it has been. Totally agree that Resolve processes everything in 32-bit float, but the 17 or 33x .cube LUTs people have used and distributed for many years are still subject to the limitations of integer math. That's why (D)CTLs have become popular. However, I could be incorrect. I know someone I can ask, but @Rohit Gupta and @Nick Shaw would also know.

Also, I'm not clear on your background, so apologies if you're well-versed on all of this already.
 
but the 17 or 33x .cube LUTs people have used and distributed for many years are still subject to the limitations of integer math.
I don't think this has been the case for at least 10 years. I believe most integer LUT formats are an artifact from the CPU processing days. Open any .cube file in a text editor and you'll see float values in the 0-1 domain. I doubt that industry standard software like Resolve would down-convert those floats to integer, that would be incredibly inefficient as it would require a second round-trip back to float before uploading the LUT data to the GPU. 99.9% sure no serious software does that... well except Photoshop but that's another story.

I might totally miss something here as I haven't worked on Resolves codebase but that's generally not how it works. There are of course other, legitimate use cases for DCTLs that I mentioned in my previous reply and most of it comes down to control over function parameters and the ability to encrypt your code.
 
I don't think this has been the case for at least 10 years.

The concept is pretty deeply embedded in the color / color science community. You can find articles referring to it here:

"Working in this native linear space ensures the cleanest, consistent transform possible. And because it doesn’t break 32bit float like LUTs do, theres no clipping or clamping of data. It’s completely non destructive and can even be fully reversed with zero loss in quality."

...here:
"LUTs are destructive, in two ways; First, their sample-based functionality means they can’t be perfectly reversed, because they weren’t perfectly applied. Key sample points were analyzed and modified, and the blanks were filled in by mathematical estimation; Second, any values falling outside the range of the table are clipped, which can lead to loss of shadow or highlight detail."

and here:
"The default Range of 0 to 1 is standard for all LUTs, and is the reason for clipping when it occurs. A LUT will only interpret values within that range (anything below 0 becomes 0 and anything above 1 becomes 1), but has no limit to the range of what it can output."

However, this would not be the first time a widely-held notion has been dispelled, so there could be a large hole in our common knowledge and we could all still be wrong. I've spent all evening reading through various papers and to be honest it's been hard to find a scientific reference that pins down exactly what we're discussing, especially the concept you pointed out of a 32-Bit Float LUT that uses values from 0-1. I have seen that formatting before, but it never registered with me that those are in fact not integer values. If you are correct, this will be somewhat of a revelation.

So, probably time to RTFM.... (Pg 3145)

"It’s well known that when a 3D LUT is fed values that are outside of the range that LUT is designed to handle, the out-of-range data will be clipped. Since many LUTs are designed with digital cinema workflows in mind, the practical result is that feeding a video signal with super-white in it to a 3D LUT that’s designed for full-range data (0–1) will clip the super-white part of the signal."

That passage has been in the Resolve manual for a long time. Maybe they are referring to integer-based 10- or 12-bit luts. If 32-Bit Float encoded luts didn't clip anymore that would be a huge help.

Let's test it in Resolve.

I created a simple node tree, using an ARRIRAW clip from ARRI's website.

Node 1: The shot with no correction
Node 2: The shot pushed into heavy clipping with the LOG Offset wheel.
Node 3: A node with a simple 32-Bit Float ARRI LogC to 709 .cube LUT
Node 4: A node that attempts to recover the highlights pushed into clipping by Node 2.

The shot with no corrections
Image 1.jpg


Node 2 Enabled: pushes the shot into heavy clipping using the LOG Offset wheel
Image 2.jpg


Node 4 Enabled: it recovers the shot and completely removes the Offset added in Node 2
Image 3.jpg


Node 3 Enabled: a 32-Bit Float cube LUT is placed between the clipping and recovery nodes and causes the clipping to return. This is what integer processing does in a float pipeline with values outside of its range.
Image 4.jpg

I've attached the 32-bit float LUT and a DRP to the project below. Here is the link to the ARRIRAW shot (M002C005_161207_R00H.mxf). If anyone wants to take a look, please let me know if I am doing anything incorrectly. You can use a different test clip if you like, just make sure its a camera raw file.
 

Attachments

  • CUBE LUT and DRP.zip
    297.8 KB · Views: 2


The concept is pretty deeply embedded in the color / color science community. You can find articles referring to it here:

"Working in this native linear space ensures the cleanest, consistent transform possible. And because it doesn’t break 32bit float like LUTs do, theres no clipping or clamping of data. It’s completely non destructive and can even be fully reversed with zero loss in quality."

...here:
"LUTs are destructive, in two ways; First, their sample-based functionality means they can’t be perfectly reversed, because they weren’t perfectly applied. Key sample points were analyzed and modified, and the blanks were filled in by mathematical estimation; Second, any values falling outside the range of the table are clipped, which can lead to loss of shadow or highlight detail."

and here:
"The default Range of 0 to 1 is standard for all LUTs, and is the reason for clipping when it occurs. A LUT will only interpret values within that range (anything below 0 becomes 0 and anything above 1 becomes 1), but has no limit to the range of what it can output."

However, this would not be the first time a widely-held notion has been dispelled, so there could be a large hole in our common knowledge and we could all still be wrong. I've spent all evening reading through various papers and to be honest it's been hard to find a scientific reference that pins down exactly what we're discussing, especially the concept you pointed out of a 32-Bit Float LUT that uses values from 0-1. I have seen that formatting before, but it never registered with me that those are in fact not integer values. If you are correct, this will be somewhat of a revelation.

So, probably time to RTFM.... (Pg 3145)

"It’s well known that when a 3D LUT is fed values that are outside of the range that LUT is designed to handle, the out-of-range data will be clipped. Since many LUTs are designed with digital cinema workflows in mind, the practical result is that feeding a video signal with super-white in it to a 3D LUT that’s designed for full-range data (0–1) will clip the super-white part of the signal."

That passage has been in the Resolve manual for a long time. Maybe they are referring to integer-based 10- or 12-bit luts. If 32-Bit Float encoded luts didn't clip anymore that would be a huge help.

Let's test it in Resolve.

I created a simple node tree, using an ARRIRAW clip from ARRI's website.

Node 1: The shot with no correction
Node 2: The shot pushed into heavy clipping with the LOG Offset wheel.
Node 3: A node with a simple 32-Bit Float ARRI LogC to 709 .cube LUT
Node 4: A node that attempts to recover the highlights pushed into clipping by Node 2.

The shot with no corrections
View attachment 10549


Node 2 Enabled: pushes the shot into heavy clipping using the LOG Offset wheel
View attachment 10550


Node 4 Enabled: it recovers the shot and completely removes the Offset added in Node 2
View attachment 10551


Node 3 Enabled: a 32-Bit Float cube LUT is placed between the clipping and recovery nodes and causes the clipping to return. This is what integer processing does in a float pipeline with values outside of its range.
View attachment 10552

I've attached the 32-bit float LUT and a DRP to the project below. Here is the link to the ARRIRAW shot (M002C005_161207_R00H.mxf). If anyone wants to take a look, please let me know if I am doing anything incorrectly. You can use a different test clip if you like, just make sure its a camera raw file.

Thanks Jason for this great post!

This actually confirms what I know about LUTs in my experience using them. You could describe them as ‘destructive’ in a sense.

I’d like to see the same test done with a dctl for comparison.

Maybe, at first sight, when using a LUT at the end of a node tree, it’s not such a big deal. But when using a LUT somewhere in the middle, for example in a color managed workflow that aims to deliver multiple output formats (sdr, hdr etc), it might be significant to have all data till the end of the pipeline.
 
I think it's just confusing terminologies. My question was:
What does "float" mean in colorist terminology? Below 0.0 and above 1.0?

And it sounds like that is indeed what you're describing here:
The default Range of 0 to 1 is standard for all LUTs, and is the reason for clipping when it occurs. A LUT will only interpret values within that range (anything below 0 becomes 0 and anything above 1 becomes 1), but has no limit to the range of what it can output.

A more precise definition would be: A LUT contains values within a certain number range or domain. That domain might be between 0.0 and 1.0, which is the default for many LUTs, or it might be between 0.2 and 0.8 or anything else. Most LUT interpolation implementations will "clip" values below the domain min and above the domain max because extrapolation beyond these points would be hard to get right and computationally quite expensive in real-time applications without a pre-processing step. (It's definitely possible to do this technically but it's not as trivial as a simple 3d texture lookup call in openGL which is how most implementations work.) A better reference than research papers here is to just look at code as source of truth. The OpenColorIO Github repo has color lookup implementation functions for 2D and 3D textures for anyone to study but it honestly isn't all that wild.

That said, none of this has anything to do with floating point or integer number formats.
Just to illustrate, an 8-bit integer LUT with points [0, 128, 255] is exactly as precise as its 32-bit floating point equivalent with points [0.0, 0.5, 1.0]...
And because it doesn’t break 32bit float like LUTs do
LUTs don't break 32bit float. Lookup functions clip their output image data to the domain min and max of the lookup table they apply to the input image data. What you're observing in the images you posted is a domain clipping issue but it has nothing to do with integer vs floating point number formats or the interpolation accuracy of LUTs.

LUTs are destructive, in two ways; First, their sample-based functionality means they can’t be perfectly reversed, because they weren’t perfectly applied.
That is true for many LUTs because of how the data points they contain were derived but definitely not an inherent technical shortcoming of lookup tables.
There are many aspects of color grading, mostly relating to I/O, that do need an inverse function for workflow reasons, but there are also many aspects that absolutely don't. What's the inverse of a lift gamma gain modifier with a subsequent printer lights adjustment? I'm not suggesting there isn't an inverse, it's all just math, but it simply doesn't matter. Whether lift gamma gain is functionally evaluated at runtime or whether the result of its function is stored in a lookup table with sufficient density to capture its transform characteristic does not matter with regards to the quality of the final image and I would challenge anyone to prove me different. The problem is something else and I will get to that at the bottom of this post.

Key sample points were analyzed and modified, and the blanks were filled in by mathematical estimation;
That is also true, but I think the point is futile and here's my reasoning: "Estimation" is really interpolation in this case. And pretty much all of applied color science is interpolation in one way or another. For example in the form of continuous spline interpolation. A segmented spline is nothing but a continuous evaluation of points along a curve, between points that define the beginning and end of the spline segment¹.That is interpolation between two or more points, just like color lookup functions use interpolation to compute points in between given data points in three-dimensional space.

What I think the real argument is:
LUTs are just data containers and there are no standards or restrictions that define how the data points are collected, distributed or interpolated¹.
Many LUTs are created using interface tools, where the individual color space transformations that are baked into the LUT have no spatial awareness of each other. In other words, nothing is stopping you from baking a 100+ node tree that is comprised of anything from I/O CSTs, transforms in multiple color spaces, completely unorthodox color warper transformations (because a single shot needed that extra boost of dark, saturated yellows) and a trusty old 2383 print LUT to top things off, into a single LUT. And even that is perfectly fine if you're going to be using that LUT as a color transfer medium to a third party application or platform. Given the LUT has high enough density to capture the intricacies of the grade, and you can ensure you're not clipping shadows or highlights, there is not technical disadvantage that will be apparent in the final image. When it does and when it doesn't make sense to use LUTs workflow-wise is another question, of course.

What I think the solution is:
Don't use LUTs for I/O or when you don't know the exact input and output color space a LUT was designed for. Don't use LUTs if you need accurate inverse transformations. Don't use LUTs when you want to make changes to individual parameters later. Don't expect LUTs to magically retain color smoothness when its transform data points are the result of un-smooth functions (which can easily result from combining different tools in a grading software, even if the individual tools produce smooth transforms).

To bring this discussion at least somewhat back on topic, all of this is a long winded explanation of why I'm creating Color.io.
It solves the I/O via ACES and helps designing re-usable looks that sit in between these color management layers. The color engine that is responsible for creating the look transforms retains much higher color smoothness than most tools I've used. It does this primarily by using custom color models that are much more appropriate for creative color grading than cone models and by using bidirectional parallelizations that make the individual tools "aware" of each other in the sense that, for example, the luminance modifier "knows" how much density was applied both in a previous adjustment and how much density is going to be applied in a subsequent adjustment in the processing chain. Most tools are actually "bidirectionally aware" in more than one color dimension, to retain an overal smooth 3D transform shape. This is also the reason Color.io can not be a plugin to an existing software because it requires its own processing architecture that I've written from the ground up to achieve real-time performance. That's all really just a more technical explanation for why @Aaron Kessler probably finds it easier to get nice looking images with the tools in the app. You'd have to jump through so many hoops and be ultra careful every step of the way to get these things right with existing tools. Maybe that makes it a "weird tool that is super niche", but I may just be a bit of a weirdo for obsessing over these things so much. But it's keeping me busy. :D

¹ Depending on the spline algorithm there may be additional weights that determine the slope of the interpolated curve (think: bezier handle points)

² Tetrahedral interpolation has actually become somewhat of an interpolation standard for LUTs. Small anecdote: I had a back-and-forth with Blackmagic in 2014 and could finally convince them to implement tetrahedral interpolation for color lookups in Resolve.
 
Last edited by a moderator:
Thanks Jason for this great post!

This actually confirms what I know about LUTs in my experience using them. You could describe them as ‘destructive’ in a sense.

I’d like to see the same test done with a dctl for comparison.

Maybe, at first sight, when using a LUT at the end of a node tree, it’s not such a big deal. But when using a LUT somewhere in the middle, for example in a color managed workflow that aims to deliver multiple output formats (sdr, hdr etc), it might be significant to have all data till the end of the pipeline.
You need to know the ins and outs when embedding LUTs into a pipeline. It's got nothing to do with interpolation or technical limitations of LUTs. It's just that they don't have an interface that tells you "Hey, I was designed for Log-C input and will output Davinci Wide Gamut RGB". You would get a visually indistinguishable result if you would embed a DCTL that clips your values as part of its function evaluation (same with CST, ACES transforms, or a UI curve adjustment that clips your highlights)
 
Back
Top