MomsSpaghetti

Ah, okay. I see horde-safety kinda like integrates the Fedi code in their filters.

Fedi-/Horde-Safety use an image interrogator, to query what’s actually in the image. That’s immune to that. They also have all these heuristics. (You can have a look in their code.) It’ll come with false-positives, though. It’s a trade-off.

Huh, now that's pretty interesting. I will defiantly take a look into their code, I want to see how they programmed it.

Mind you’re currently in the process of fleshing out your own interpretation of the wheel. My recommendation was not to do that. And your approach has the usual downsides: People can use other words, synonyms. Describe it. Or leverage wrong spellings. Jam an adjective in-between. Do an exact request, but in Chinese language… All that will circumvent filtering by simple word filters. And that’s usually what happens 5 minutes after you roll out such a system. It’s a) pretty much not effective at all. And b) it’s complicated and you’ll end up in a cat and mouse game.

Yeeaaaahhhh...I just realized that that now...emphasis on the process of fleshing out my own interpretation. I wouldn't recommend as something to actually make and integrate as censorship. I think I was literally doing that, trying to showcase how I thought how the censorship/filtering worked, like the basics of it...I knew it wouldn't be as accurate or effective because not all censorship/filters actually work like that...I guess in my head it was more of a "demonstration" rather than an actual tool/solution. I think I meant to demonstrate how the censorship/filter on what words and associations to focus on instead of banning innocent prompts because of too much censorship (Unfortunatly, that what Grok does. Because they effed up knowing releasing the ability someone's post without consent and now it all backfired, cranking up the censorship and making it practically unusable. And the fact the government is enforcing it also makes it worse) ...But you are 100% right: its not effective since people can use ways to circumvent the filter like you said, use different words, synonyms, mispellings, sentences, etc. (Man, I wouldn't be too surprised if it was less than 5 minutes...its insane...). Very ineffective and complicated...I just thought about its trying to keep adding words when people can bypass is hard as hell. And the cat and mouse game? God, couldn't be more true. Trying to add more words to the filter and people STILL find ways to bypass would be exhausting.

I generated these with my prompt. I think this is what you were going for. Some of them were "cherry picked" as my way of prompting doesn't always work, but these were absolutely generated with my prompt, many of them actually (when I increased the numbers of images). I used some of the AI image model's (a random generator from Perchance) settings like guidance and art style (didn't really help much until I put real human being/photograph in the prompt...bruh moment). No flyaways, no buns :3

I can share a erm vanilla version of the prompt, should get the about the same result:

woman with long and tall ponytail that sits up high on top of head, the ponytail is pulled-back-tightly/slicked back with gel and stretches the forehead. The woman is a real human being. She is in a professional photograph. Avoid styles, increase guidance (try 12-15), and increase number of images (10 is okay, 20 is good, 30 is great, 40-60 is amazing!)

I'm thinking like (for programming filter only):

contains/includes/with = (words go here) exclude/omit = (words go here)

Now, you can't include everything at first, just keep adding and refining the filters over time or from reports from users. Very minor idea:

Words (adjective, noun): little, young, small, tiny Containing/including with (nouns): boy, girl, child, kid

(can even add suffix/prefix for like pre-teen, pre-adolescent, etc)

Exclude/omit from (adjective, verb, adverb, preposition): [you get the idea]

if there's a word with (-s, -es, -ed) would also include that too...

This tells the filter to what to scan or analyze for. Whether the words are categorized or not (doesn't matter) the core concept remains the same. I think there may be similar pre-made solutions or tools on github like my idea, I would also look into those. Its not as hard as you think, just a bit time consuming to configure...

Yeah, I think something like that would be very useful and helpful. Its easier to use pre-made tools or features and integrating in your project or service. Consulting other projects is a smart thing to do because they can give you some insight about filters and censorship, stuff like. Filter for the Fediverse, like this one: https://github.com/db0/fedi-safety? Seems interesting, I also remember AI Horde, I think I used it a couple of times. It's great their filtering works as intended. I think the dev should take this advice, and maybe even hire or consult somebody to help him with this. I am not programmer or coder (I can't do this shiii without Microsoft's Copilot, it was the one helping with all the coding and troubleshooting lol. ) but maybe I can make a "filter" example, albeit a simple one, just as an example. I'd rather have filtered out words than Grok's horrible use of censorship, blurring images that's not even NSFW...

I kinda wanna aim for:

(child) ≠ [forbidden word]

or

(little\young\small) | girl/boy/child ≠ [forbidden word] Note: No specific order. Just the words contained in ( ) with (like girl/boy/child) to let the filter know to filter those forbidden words out those in combination of the other words.

(of course not the only one)

Filter system shouldn't censor innocent prompts, but rather specific ones based off the instructions its given. Child ≠ forbidden word example I've provided is a good way to add censorship without blurring images that uses the prompt: Two adults, Sally and Tom lays in bed together, wearing underwear. Sally wears bras and panties and Tom is shirtless in boxers. A prompt like this shouldn't be censored because Sally and Tom are adults, yet why is being censored anyways? If it allows NSFW as long as the adults are characters, then it shouldn't be an issue. This was Grok's mistake: allowing NSFW but had to enforce stricter censorship because the dumbahh developers allowed Twitter users to edited pictures in posts and creeps and pedos made sexualized images of women and children. Now Grok users can't generate any NSFW content involving consensual, legal adult characters only, after being marketed as "unrestricted/uncensored" type AI image generator. This is the type of issue we want to avoid. Perchance allows NSFW and not have this "family friendly" or "all ages" image. So allow NSFW content for any prompt containing adult-aged characters only. Shouldn't be too hard...

Yeah, to be honest, I don't the dev planned this out before adding AI stuff. He might have been as aware at the time. That was probably a mistake on his end. (Not me having to look up the meaning of drivel 😭😂 ...I admit sometimes I write too much and most of the time it I don't understand my own writing...I see as nonsense and stupid as well, so I don't blame you and completely understand lmao. I just yap too much, ChatGPT ahhh writing 🥀Sorry about that... 😅). I agree the creators does need to be held accountable for this, since he didn't add any safety measure...I don't mean to blame the users for this NSFW issue, I was just saying the are the ones who are in charge of their own generators (because thats how Perchance was designed, there wasn't really a universal report system), its not their fault for this whole NSFW debacle. I am hoping the dev can solve this NSFW w/kids issue somehow...

That's not the dev's responsibility; its the users who create these generators. They are the ones who code these options into the generators since they created the generators. Perchance generators rely heavily on coding. The dev never coded preteen or teen options into this generator, the creator did. And its hard to tell who the hell the creator is... Sadly, with the way Perchance was designed, these creators could get away with this type of stuff because there isn't a way to report them based off of username...for now only OP could email the link to the generator for the dev to take down. Honestly, its effed up the creator even added preteen girl and teen girl as options with adult woman and mature woman, well knowing their generator is NSFW/18+. I have never seen options like that in those types of generators, it was always 18+. Dear god, that creator is honestly sick and messed up in the head if they want to create that type of content with children in it...What the hell...

I think people seem to forget:

• This is a public website. Users can create and post their own generators. Often, these users are in control of the censorship, moderation, and safety of their generators, not the dev of the website. Its not solely his responsibility. Perchance is not a large corporate website; its a individual-made website driven by the community and users. It was never designed with universal safety like that to begin with.

• Its ran and maintained by one person, who's already burning his money to keep the site up and running it while paying for these AI models. He has limited resources. Implementing stronger safety measures takes time, money, and resources. And this website is completely free! He makes money through ads for the AI image models (or model, I think he only uses one).

• It's going to be nearly impossible for him to add a universal safety system for all these user-made generators, not without more resources and money. He's making very little from the website with ads. He can't do it at the moment give him some grace. Its hard to combat inappropriate images with limited resources. Same principle with Discord: they have a report system so they can handle problems but ultimately its their responsibility to the discord mods from user-made servers to manage rules, moderation, safety regulation, and reports on their server. Same with Reddit, you can create communities based off of an interest or topic. The mods are the ones in charge of all the rules and moderation. Reddit only handle reports submitted by people.

Perchance, like Discord and even Reddit, is a community/user-driven platform. The users are in charge of the things they create, not the developers of the platform. The developers can't oversee and deal with individual user created stuff. Do you think it will be easy for the creator to add some sort of rule, moderation, safety, or censorship issue to all these generators? No. It will take a very long time, especially with limited resources. As far as the AI image model generating illegal things such as the CSAM/CP, he would need to find a way to add censorship for the model. He can't inject it into the model; he has to code and program the censorship feature. But he'll have to import it into the ai image model plugin but he can't without building the censorship feature first. Platforms like Grok or Microsoft's image generator already implemented censorship. It'll blur or show a message the user cannot generate the content because its against the rules/inappropriate. He has to brainstorm how to design it, code it, program it, implement it, then start develop it from scratch, figuring out how to censor the images, words or prompts, figure out how to code and program it into the AI image model, texts or generators, on top of that—little funds, making some money only but running ads for the AI image models, a public community/user-driven website, thousands of individual-made generators,

ONE person paying and maintaining the servers, the domain, AI image models and AI generated text, functions, coding, and programming of the website and its features, burning all that money for those resources all by himself without making little to no money...it's not going to be as simple as you think. Censorship takes a lot of development: it isn't just coding few words into the AI and call it a day. No, you have to build and develop it, then implement it into the system. And he might already have plans or working on the features, improvements, bug fixes, etc. He just completed a migration. He will need time to work on this especially if the website was never designed or implemented such complex features like this. Good lord.

Yeah, its working for me now too...well, it did earlier so migration is nearly or fully completed I believe...

(I sworn I commented on this reply...eh whatever) There's no pedophilic plugin; the creators code and implement the options into their generators. They program these options as "prompts" for the AI image model to generate. So the AI image model is able to generate it. Its still just as unfortunate.

It's not a bug. You are using an external AI image generation model and they don't come with built-in censorship/moderation. Censorship (or even moderation) is a feature you have to implement into the system, you can't implement it into the AI image model. Usually, censorship is based off of words used in a prompt or maybe how the words are used in the prompt. If you didn't add a censorship or moderation feature into the system, then its going to produce CSAM/CP since there's no filter or guards. People have reported innocent prompt being rejected because the AI thinks the prompt is inappropriate when its not. Like:

• Sally, a curvaceous and busty woman, wears a bikini at the beach (Automatically rejected)

• A man is uses a gun fires to fire a bullet at a zombie head and blows its brain (Automatically rejected)

• A mother breastfeeding her newborn baby (Automatically rejected)

• A guy punches another guy's face, leaving it bloodied and bruised (automatically rejected)

These examples are either consensual/legal adults, non-extreme violence, a mother doing a natural thing for her baby, or fictional violence regarding fictional creatures. However these types of prompts still get rejected because the censorship thinks the words contained in the prompt or how the prompt is written in those words are inappropriate even though its not? That's poor censorship and moderation. Normal stuff like this shouldn't be censored, instead stuff like this should:

extreme pornographic/18+/sexual (terms describing extremely sexual actions, behavior, poses, positions, erm...bodily functions..., sexualized partial or full nudity, extremely skimpy or revealing outfits, body parts, effects, nonconsensual acts, actions, behaviors as well as sexual violence and abuse like SA, rp, etc) associated with children under 18 (newborn, infant, toddler, child, kid, preteen, teen, adolescent, <18 years/years old/aged/age or like aged 9 or age 9, little girl, little boy, young girl, young boy, you get the point) and even animals. That type of stuff. AI generated CSAM/CP is also illegal now although it wasn't before because Grok allowed people to edit users' posts containing photos without their consent and creeps and pedos were using it to generate horrible sexualized images of women and children (specifically young girls under 18).

Obviously, sexual violence and abuse is disgusting and immoral for any one of any age or gender...but if its fictionalized with fictional adult characters, that's up for you to censor or not...

I am not sure how feasible it would be for you to implement a censorship or moderation features at the moment...that's beyond something for a public community website like Perchance and a solo site runner like you. You need a lot of resources and money for that...as well as multiple moderators or staff helping you maintain that stuff...You probably won't be able to implement some type of censorship for children for quite some time or any other type of safety features...to be honest, a lot of this censorship and safety features is the creator's responsibility since they're the ones who made these and there wasn't any universal report/moderation/safety features to begin with. I would suggest taking your time coming in researching and coming up with a way censor harmful and illegal images of children. This may take a long while but it'll be worth it. I think the best bet for now is someone reporting these issues to you, whether there's something wrong with the AI image model, generators, text, etc. And if you can, the most simplest way for safety, is adding a basic report button. To flag down any illegal content someone or something is producing. The gallery, for now, is the best place to catch someone producing and posting AI gen CSAM/CP. I wish you the best of luck, man. This type of stuff ain't easy, even for larger websites ran by multiple people.

Yes, but unfortunately, that its not the only type of AI image generator either…There’s some with the option of changing the age of your subject/character…I have seen a few of those type of generators…However, not all of these generators have the intention of wanting to produce CSAM/CP material. Some of them just want to include different ages/age groups for more variety and diversity. It can definitely be hard to tell whether the intentions are innocent or note; because you have different age groups containing underaged/minors (babies, toddlers, children, preteens, and teens) grouped in with adult age groups (young adult, mature/middle-aged adult, and elder/senior) and adult content for a wider range for character options or to produce CSAM/CP? If the adult content is extremely NSFW, then I agree that children shouldn’t be included at all. That’s borderline CSAM/CP.

In this case, it's been made very clear what the intention of this generator is. Preteen Girl and Teenage Girl included in gender-specific age groups is a huge red flag. Usually, with those types of generators, it's 18+...so I would strongly advise you to report that.

I think people seem to forget that this website is more of a passion project and community/user-based. Users are allowed to create, run, maintain, and moderate their own generators, rather than just multiple moderators utilizing a universal safety system. Trying to implement universal safety features for individual-ran generators probably wouldn't be very feasible right now. People seem to either forget or not know this website is run by one guy…He probably doesn’t have the money and resources to implement an enforced and stricter safety system and features. I don't think he even designed Perchance in mind like that and I mean it's not entirely his responsibility to oversee the safety and rule breaking of these generators; it's the creators job to do, they have the ability to do these themselves.

…Moderators, auto–moderation, auto-banned words/associations, report system, scanning/screening, wait time, pre-approval, requirements, background checks, task requirements, etc are not easy safety features to implement for one guy running. And there’s so many user-created generators, it would be hard trying to moderate and flag them down. On top of that, trying to stop people from bypassing it and there’s also people who generate this stuff for self-gratification and keeping it to themselves (by not posting it online) which can’t be done much with that…Now you can’t stop people from doing that. The best he could do is add a basic report button. I did see him mention it shouldn’t be possible to generate CSAM/CP if it's a bug in the image generator. I don’t really think it is a bug per say but a lack of censorship, more likely not censorship being implemented into the system, because if you are using an AI image model, then it can produce this stuff because it doesn’t come with built-in censorship. Censorship is associated with words used in the prompt that's deemed inappropriate. You have to add censorship as a feature. So it's not an actual bug, just a feature not implemented yet.

Now, it can be difficult to decide what’s inappropriate or not. As the dev mentioned, there is semi-nudity associated with children. Just nudity as a whole isn’t inherently sexual or bad in nature; many men, women, and children have been nude from dawn of time till now, semi nude, shirtless/topless, etc because there is a primitive/ancient, semi-primitive/ancient, or modern tribal/ethnic groups, culture/customs/traditions, environments, different cultures and places that normalize this in the modern or carry on custom and traditions, historical/ancient/prehistoric reasons, yada yada. This is a difficult grey area to figure out. And then there’s overly sensitive censorship which will completely blur or censor an innocent prompt.

@perchance@lemmy.world Maybe you can add some type of AI chatbot thingy (safety analyzing chatbot), since AI is already so relevant, that analyzes the the whole prompt rather than just word-based? It will still analyze the words but will also analyze the intent and context of the prompt. If the user is typing an inappropriate prompt containing children with pornographic/sexual terms/words, then it will auto-reject the prompt. Or if there's a prompt written in a very suspicious way, it will request the user to rewrite, reedit, revise or auto-generate alternative prompts that's safer and more appropriate. I'm also thinking if the AI thinks something is inappropriate or suspicious but its not the user could chat with the AI clarifying their prompt and the AI will understand and allow the user to proceed with the prompt. However, the AI will reject anything that 18+/pornographic/sexual content associates children/teens/minors/underaged (you get the point) or other illegal things like zoophilia or animal sa. Just the concept of just chatting with the AI safety analyzing bot (what I'm calling it for now) like you would do with ChatGPT. However, the bot would have to be associated with the AI image generation textbox so it could analyze the prompt. It solves the problem with inappropriate prompts and overly sensitive censorship.

I think this would more likely be feasible for you with the ai chat text whatever plugin but consider how you would implement it universally.