restore and edit a natural image

[ohad204]

Hi,

For my project, I'm trying to edit a natural image.
However, I noticed that the edited image looks nothing like the original.

Even when I try to restore the original image using the same prompt, the result is still completely different.
Could you help me understand what's going wrong?

The code:

import torch
from diffusers import FluxPipeline
import matplotlib.pyplot as plt
from PIL import Image

pipe = FluxPipeline.from_pretrained(
"black-forest-labs/FLUX.1-dev",
torch_dtype=torch.bfloat16,
custom_pipeline="pipeline_flux_rf_inversion")
pipe.to("cuda")

image = Image.open("_DSC9191.JPG")

new_width = 3312
new_height = 3312
width, height = image.size # Get dimensions
left = (width - new_width)/2
top = (height - new_height)/2
right = (width + new_width)/2
bottom = (height + new_height)/2
image = image.crop((left, top, right, bottom))

inverted_latents, image_latents, latent_image_ids = pipe.invert(
source_prompt='garden of flowers',
image=image,
num_inversion_steps=28,
gamma=1.0
)

edited_image = pipe(
prompt="garden of flowers",
inverted_latents=inverted_latents,
image_latents=image_latents,
latent_image_ids=latent_image_ids,
start_timestep=1/28,
stop_timestep=7/28,
num_inference_steps=28,
eta=0.9,
).images[0]

plt.imshow(edited_image); plt.show()

The image:

[LituRout]

Hi @ohad204, thanks for using RF-Inversion in your project. As I see from your code above, you are using "garden of flowers" while inverting the image. Ideally, you don't want to use that unless you want the features corresponding to "garden of flowers" removed from the original image. Can you try using null prompt during inversion as discussed in Section 4 https://openreview.net/pdf?id=Hu0FSOSEyS? Also, try increasing the start and stop time steps if you want exactly the same image. For quick experiments, you can try using our demo: https://huggingface.co/spaces/rf-inversion/RF-inversion

I used start=10, stop=20, and eta=1 to get the following result. Feel free to tune for better reconstruction or editing.

[ohad204]

Thank you very much!
What parameters would you recommend using in the code to achieve this result?

Do you think Flux can handle images with an aspect ratio different from 1:1, like this photo?
I would like to adjust the width and height.

I tried it again with the color specified, and it changed the trees to red.
How would you suggest dealing with that?