PATENT: Optical Illumination Mapping

Techniques for augmenting an appearance of a first object. Embodiments include capturing a visual scene for display. Here, the visual scene includes a physical object and wherein the visual scene is captured using one or more camera devices. The physical object is identified as a first redetermined object type, based on one or more object identifiers associated with the physical object. Embodiments also retrieve visual characteristics information corresponding to the first predetermined object type. A sequence of frames that includes the first object is then rendered for display, where the appearance the first object in the rendered sequence of frames is augmented based on the retrieved visual characteristics information and an appearance of the physical object in the captured visual scene.