Yes, but unfortunately, that its not the only type of AI image generator either…There’s some with the option of changing the age of your subject/character…I have seen a few of those type of generators…However, not all of these generators have the intention of wanting to produce CSAM/CP material. Some of them just want to include different ages/age groups for more variety and diversity. It can definitely be hard to tell whether the intentions are innocent or note; because you have different age groups containing underaged/minors (babies, toddlers, children, preteens, and teens) grouped in with adult age groups (young adult, mature/middle-aged adult, and elder/senior) and adult content for a wider range for character options or to produce CSAM/CP? If the adult content is extremely NSFW, then I agree that children shouldn’t be included at all. That’s borderline CSAM/CP.

In this case, it's been made very clear what the intention of this generator is. Preteen Girl and Teenage Girl included in gender-specific age groups is a huge red flag. Usually, with those types of generators, it's 18+...so I would strongly advise you to report that.

I think people seem to forget that this website is more of a passion project and community/user-based. Users are allowed to create, run, maintain, and moderate their own generators, rather than just multiple moderators utilizing a universal safety system. Trying to implement universal safety features for individual-ran generators probably wouldn't be very feasible right now. People seem to either forget or not know this website is run by one guy…He probably doesn’t have the money and resources to implement an enforced and stricter safety system and features. I don't think he even designed Perchance in mind like that and I mean it's not entirely his responsibility to oversee the safety and rule breaking of these generators; it's the creators job to do, they have the ability to do these themselves.

…Moderators, auto–moderation, auto-banned words/associations, report system, scanning/screening, wait time, pre-approval, requirements, background checks, task requirements, etc are not easy safety features to implement for one guy running. And there’s so many user-created generators, it would be hard trying to moderate and flag them down. On top of that, trying to stop people from bypassing it and there’s also people who generate this stuff for self-gratification and keeping it to themselves (by not posting it online) which can’t be done much with that…Now you can’t stop people from doing that. The best he could do is add a basic report button. I did see him mention it shouldn’t be possible to generate CSAM/CP if it's a bug in the image generator. I don’t really think it is a bug per say but a lack of censorship, more likely not censorship being implemented into the system, because if you are using an AI image model, then it can produce this stuff because it doesn’t come with built-in censorship. Censorship is associated with words used in the prompt that's deemed inappropriate. You have to add censorship as a feature. So it's not an actual bug, just a feature not implemented yet.

Now, it can be difficult to decide what’s inappropriate or not. As the dev mentioned, there is semi-nudity associated with children. Just nudity as a whole isn’t inherently sexual or bad in nature; many men, women, and children have been nude from dawn of time till now, semi nude, shirtless/topless, etc because there is a primitive/ancient, semi-primitive/ancient, or modern tribal/ethnic groups, culture/customs/traditions, environments, different cultures and places that normalize this in the modern or carry on custom and traditions, historical/ancient/prehistoric reasons, yada yada. This is a difficult grey area to figure out. And then there’s overly sensitive censorship which will completely blur or censor an innocent prompt.

@perchance@lemmy.world Maybe you can add some type of AI chatbot thingy (safety analyzing chatbot), since AI is already so relevant, that analyzes the the whole prompt rather than just word-based? It will still analyze the words but will also analyze the intent and context of the prompt. If the user is typing an inappropriate prompt containing children with pornographic/sexual terms/words, then it will auto-reject the prompt. Or if there's a prompt written in a very suspicious way, it will request the user to rewrite, reedit, revise or auto-generate alternative prompts that's safer and more appropriate. I'm also thinking if the AI thinks something is inappropriate or suspicious but its not the user could chat with the AI clarifying their prompt and the AI will understand and allow the user to proceed with the prompt. However, the AI will reject anything that 18+/pornographic/sexual content associates children/teens/minors/underaged (you get the point) or other illegal things like zoophilia or animal sa. Just the concept of just chatting with the AI safety analyzing bot (what I'm calling it for now) like you would do with ChatGPT. However, the bot would have to be associated with the AI image generation textbox so it could analyze the prompt. It solves the problem with inappropriate prompts and overly sensitive censorship.

I think this would more likely be feasible for you with the ai chat text whatever plugin but consider how you would implement it universally.