Forum:BioStars and ChatGPT
1
5
Entering edit mode
7 weeks ago
Jack Tierney ▴ 280

Today StackOverflow have temporarily banned the use of GPT based answers (see here). People who are suspected to be posting ChatGPT answers without vetting them will get sanctions. Perhaps biostars should follow suit and place a notice along the same lines.

For example, take the most recent post at the time of writing. ChatGPT returned the answer attached in screenshots. This may not be the best example but it was quick to produce and would detract from the valuable help that is currently being provided in the replies.

Regardless, I'd be interested to hear what others think...

Image showing a generally unhelpful biostars answer

ChatGPT Meta GPT • 1.2k views
ADD COMMENT
3
Entering edit mode

I'd almost want to go the other direction and have the ChatGPT answer come up on question posting, e.g.:

1) User enters question

2) User hits [Post] button

3) Window pops up containing the ChatGPT answer and "Does this answer your question?" [Yes] [No]

4) On YES, question is posted with ChatGPT answer pre-marked as the accepted answer; on NO the question is posted alone and without a GPT response

ADD REPLY
4
Entering edit mode

That would be great! However, I suspect lower quality questions might get lower quality answers. This will disproportionately impact newer bioinformaticians who might struggle to vet the proposed answers in some unfortunate cases where the answer produces a semi-plausible looking result and not a straight error or empty file.

I can definitely see it going that way in the future for all sorts of support services though

ADD REPLY
1
Entering edit mode

How are beginners going to know if this is a good answer that actually solved their problem ? They are the very last people who could actually verify that an answer is a good and correct. ChatGPT seems to be very good at posting answers which sound and look right but are actually a load of garbage. The good thing about this community is that most people answering are experts and there are a lot of intricacies in bioinformatics which can only be addressed by experts. For that reason I don’t see this working for the current time.

ADD REPLY
0
Entering edit mode

How are beginners going to know if this is a good answer that actually solved their problem ?

Presumably by running the code and checking if the output is correct. Or is is established that beginners are so hopeless they cannot even do this?

ADD REPLY
0
Entering edit mode

That might work for simple coding questions like 'How do I merge a data frame with another?' (although the experience from Stack Overflow is that even simple questions can be got wrong). But it would fail with questions like 'What are best practices RNAseq experiments with x replicates with an over dispersed phenotype and low quality RNA.... etc' or 'What's the best algorithm for this de novo assembly....'. Bioinformatics can be very complex and context specific, so beginners (and often experts as well!) have absolutely no clue whether answers to even moderately complex questions with no simple yes/no answer are correct or not and marking them as correct on that basis could seriously mislead future readers.

This isn't meant to rip on beginners, but more to highlight how much knowledge is needed to even ascertain whether something is correct or not.

ADD REPLY
1
Entering edit mode

In my limited experience here, by far the most common case is the simple one...

ADD REPLY
0
Entering edit mode

ChatGPT is what prompted @Istvan to post this yesterday: Add artificial intelligence training related disclaimers to your profile

ADD REPLY
0
Entering edit mode

I saw that and it prompted me to make a similar discussion when I saw the news from StackOverflow. Perhaps its not a novel enough thought and should have gone in a reply to that post.

ADD REPLY
1
Entering edit mode
7 weeks ago

I think the problem will always be less pronounced here than on Stack Overflow.

On Stack Overflow people are eagerly chasing reputation points because that reputation carries an immediate value - I believe various levels of reputation open up different additional rights and permissions.

Thus, people submit suboptimal ChatGPT answers to game the system and get ahead on the ladder.

No such ladder exists on Biostar, answering with nonsense would be called out and would be detrimental to the user trying to get ahead that way.

ADD COMMENT
0
Entering edit mode

That makes sense. If the influx of ChatGPT answers is going to be relatively small compared to real answers then I imagine more experienced users will be able to point out where a ChatGPT answer goes wrong.

ADD REPLY

Login before adding your answer.

Traffic: 1524 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6