AI video generation experiments

[mjw]

Probably a significant issue is the lack of training material. AI can get (for example) someone walking down a street to look very realistic and satisfying, because there's mountains of training material consisting of people walking down a street.

But there's virtually zero material of people going for a sink in different kinds of mud or quicksand, and the overwhelming majority of training material related to sinking will always be sinking in water.

As I've mentioned, at least with Kling and Hailou, the starting image appears to be critical (more so than the prompts) in avoiding a watery ending or undesirable results.

[JSample]

Very frustrating when sand or dry-looking mud turns to liquid as they sink. I've produced gritty dry sand and liquid goo but just can't produce a good thick mud or sandy muck that acts like it no matter how much I describe high viscosity or density in prompts with water-like ripples on the surface.

But my biggest issue is random inserting of things I did not specify. I made one video and when I reviewed it I watched the sky turn dark and start showing lightning flashes even though the text made no mention of anything like that or even mention the sky or the weather. Where did that come from?

Sometimes I wonder if random stuff like that is introduced intentionally to force us to do more renders and spend more money to get useful results. Especially when I see things like bubbles on the surface of a sinking medium with happy faces on them appear out of nowhere for no reason. Say WHAT?

I apologize for not taking your frustration with the output more seriously in my initial response; my not being on your side of the production (or having spent my own money to create it) affected my perception of how you felt about it.

While I'm only generating images now, I'm familiar with the "Where did that come from?" issue. Sometimes I think the AI is programmed to be excessively "creative" in its interpretations.

I'd be frustrated too if random things like "happy face" bubbles turned up in renders I was paying directly to produce. Sometimes Microsoft Designer will generate utterly bizarre output that, while definitely showing a (fully-clothed) woman in mud or quicksand, will be in a ridiculous, comically-rendered style that looks like it belongs in a children's book (including happy forest animals sinking along with her while she has an exaggerated grin on her face). Whenever Designer does that I call it the AI "going Candyland" on me.

[BogDog]

My bad. I was referring to my AI results in general and not your work. I also have problems sometimes with ripples on the surface which are always there even if I specify no ripples or any surface movement of any kind. I am still very much a rookie at this and with the cost of AI video creation so high I may not stick with it for long.

[JSample]

My bad. I was referring to my AI results in general and not your work. I also have problems sometimes with ripples on the surface which are always there even if I specify no ripples or any surface movement of any kind. I am still very much a rookie at this and with the cost of AI video creation so high I may not stick with it for long.

Oh, I didn't think you were referring to my work; I realized that my initial response seemed to minimize or invalidate your feelings of frustration with the AI's behavior. I'm very much a rookie at this as well, although I don't face the cost of video creation (at least not now).