Robust Watermarking for Generative Images

In an article recently submitted to the arxiv* server, researchers introduced an active strategy combining image watermarking with latent diffusion models (LDM) to address the ethical concerns of generative image modeling. Their method embedded an invisible watermark in all generated images for future detection and identification.

Study: Robust Watermarking for Generative Images. Image Credit: Artography/Shutterstock
Study: Robust Watermarking for Generative Images. Image Credit: Artography/Shutterstock

*Important notice: arXiv publishes preliminary scientific reports that are not peer-reviewed and, therefore, should not be regarded as definitive, used to guide development decisions, or treated as established information in the field of artificial intelligence research.

They fine-tuned the latent decoder of the image generator, conditioning it on a binary signature. A pre-trained watermark extractor retrieved the hidden signature from any generated image, and a statistical test determined its origin. The researchers evaluated the invisibility and robustness of the watermarks across various tasks, demonstrating that the stable signature method remained effective even after image modifications.

Background

Past work in generative modeling and natural language processing has made significant strides in creating and manipulating photo-realistic images, exemplified by tools like discrete autoencoder language learner – E (DALL·E 2) and stable diffusion. These advancements have led to the development of numerous image editing tools, such as ControlNet and instruct-pix2pix, becoming popular among artists, designers, and the general public. Despite the benefits of creative artificial intelligence (AI) applications, these developments also raise concerns about the authenticity and integrity of photo-realistic images.

Embedding Signature Efficiently

Stable Signature modifies the generative network to embed a specific signature in generated images using a fixed watermark extractor. The training process involves two phases: creating the watermark extractor network (W) and then fine-tuning the LDM decoder (D) to ensure all generated images contain a designated signature via W.

HiDDeN, a classical deep watermarking method, is utilized to pre-train the watermark extractor (W). This method jointly optimizes the watermark encoder (WE) and extractor (W) to embed k-bit messages into images, which are robust to various transformations applied during training. Post-training, WE is discarded, and only W is retained. The WE produce a watermarked image by adding a residual image to the original, which is then transformed and processed to extract a message. The loss is calculated using Binary Cross Entropy (BCE), with the network architectures kept simple to facilitate LDM fine-tuning. A PCA whitening transformation is also applied to remove bias and decorrelate the output bits of W.

In the LDM, diffusion occurs in the latent space of an auto-encoder. The fine-tuning process modifies the decoder (D) to produce images containing a given message (m), which W can extract without altering the diffusion process. A fixed signature (m) is embedded in the images by fine-tuning D into Dm, using the LDM encoder to create a latent vector (z) and the decoder to reconstruct an image. The message loss is the binary cross entropy (BCE) between the extracted message and the original, while the perceptual loss controls image distortion. The fine-tuning process is rapid, taking less than a minute on a single GPU, and involves optimizing the weights of Dm over 100 iterations to minimize the combined message and perceptual loss.

Stable Signature: Effective Watermarking

The experimental results section thoroughly evaluates the efficacy of a stable signature for embedding watermarks in generated images across various generative tasks and scenarios. Firstly, the method's versatility is highlighted as it seamlessly integrates with the LDM decoder, making it applicable to diverse tasks such as text-to-image generation, image editing, super-resolution, and inpainting. Evaluation metrics like peak signal-to-noise ratio (PSNR), structural similarity index (SSIM), and Frechet inception distance (FID) are employed to assess image quality and distortion caused by the watermarking process.

Secondly, the qualitative impact of a stable signature on image generation is demonstrated through examples that show minimal perceptual differences between watermarked and original images. Despite modest PSNR values and the absence of constraints specific to human visual systems, the method effectively embeds watermarks in textured areas, preserving uniform backgrounds. This nuanced approach ensures that the embedded signatures remain inconspicuous to human observers while achieving robust watermarking.

Lastly, the robustness of the watermarking method is evaluated across various image transformations and tasks. Results indicate high bit accuracy in watermark extraction, demonstrating resilience to cropping, brightness adjustments, and joint photographic experts group (JPEG) compression transformations. Furthermore, comparisons with post-hoc watermarking methods highlight stable signature efficiency and security in embedding watermarks during the image generation process rather than afterward, ensuring minimal impact on image quality.

These findings underscore the stable signature's effectiveness in embedding robust watermarks into generated images while maintaining high-quality visual outputs across diverse generative tasks and challenging scenarios.

Conclusion

To sum up, by fine-tuning the LDM decoder, imperceptible watermarks were seamlessly embedded into all generated images while preserving the integrity of the diffusion process. This approach was compatible with various LDM-based generative models and enabled effective detection and user identification from generated images with high accuracy. It highlighted the advantages of watermarking over passive detection methods in deploying image generative models publicly. The code for reproducibility was available at github.com/facebookresearch/stable signature, and the method's environmental impact was relatively low, estimated at approximately 10 tons of CO2eq emissions for the experiments conducted.

*Important notice: arXiv publishes preliminary scientific reports that are not peer-reviewed and, therefore, should not be regarded as definitive, used to guide development decisions, or treated as established information in the field of artificial intelligence research.

Journal reference:
Silpaja Chandrasekar

Written by

Silpaja Chandrasekar

Dr. Silpaja Chandrasekar has a Ph.D. in Computer Science from Anna University, Chennai. Her research expertise lies in analyzing traffic parameters under challenging environmental conditions. Additionally, she has gained valuable exposure to diverse research areas, such as detection, tracking, classification, medical image analysis, cancer cell detection, chemistry, and Hamiltonian walks.

Citations

Please use one of the following formats to cite this article in your essay, paper or report:

  • APA

    Chandrasekar, Silpaja. (2024, June 26). Robust Watermarking for Generative Images. AZoAi. Retrieved on December 11, 2024 from https://www.azoai.com/news/20240626/Robust-Watermarking-for-Generative-Images.aspx.

  • MLA

    Chandrasekar, Silpaja. "Robust Watermarking for Generative Images". AZoAi. 11 December 2024. <https://www.azoai.com/news/20240626/Robust-Watermarking-for-Generative-Images.aspx>.

  • Chicago

    Chandrasekar, Silpaja. "Robust Watermarking for Generative Images". AZoAi. https://www.azoai.com/news/20240626/Robust-Watermarking-for-Generative-Images.aspx. (accessed December 11, 2024).

  • Harvard

    Chandrasekar, Silpaja. 2024. Robust Watermarking for Generative Images. AZoAi, viewed 11 December 2024, https://www.azoai.com/news/20240626/Robust-Watermarking-for-Generative-Images.aspx.

Comments

The opinions expressed here are the views of the writer and do not necessarily reflect the views and opinions of AZoAi.
Post a new comment
Post

While we only use edited and approved content for Azthena answers, it may on occasions provide incorrect responses. Please confirm any data provided with the related suppliers or authors. We do not provide medical advice, if you search for medical information you must always consult a medical professional before acting on any information provided.

Your questions, but not your email details will be shared with OpenAI and retained for 30 days in accordance with their privacy principles.

Please do not ask questions that use sensitive or confidential information.

Read the full Terms & Conditions.

You might also like...
AI Drives "Unbounded": Generative Infinite Game Transforms Character Life Simulations