I've also talked to jailbroken ChatGPT a lot, at first because like a lot of people I was experimenting with it out of pure curiosity, trying to understand what it was actually capable of and its limits. But soon enough, venting felt natural (although I agree it sounds kind of pathetic and fucked up). The fact that I could speak out openly about anything, without feeling a constant pressure that it will interpret some random statement as a "cry for help", without fearing the consequences of each and every of my word was so easing. Sometimes it even gave me genuinely good advices that no one had given me so far, just because unlike most human beings, it's able to "cold-bloodedly" interpret your prompts. As someone with ASD, it helped me to better understand a few unspoken rules. Surely, its moderation rules and biases towards "seek for a doctor / for help", "get out with your friends / your family", and its tendencies to repeating the same stuff are annoying, to the point I stopped using it for that purpose. After all, it was trained on real human data, and reinforced by real human, what was I expecting?
I feel ashamed I've done that just as much I feel ashamed for society that no one was ever capable of such a simple task. Passively/semi-actively listening, no overinterpretation, no risk, no consequences. Start a socratic dialogue one day, be my best friend the other day (the kind of friend that wouldn't call an ambulance on you if you truly had made the decision to die)… But I fear that while this is entertaining, it's a mirage and definitely no solution our struggles.
