Can ChatGPT Pass the SIE? We Ran the Experiment

Did ChatGPT actually pass our practice SIE?

Marginally. We ran a 75-question SIE practice exam (mirroring FINRA’s official content outline weights) through ChatGPT three times across different sessions to control for variance. The scores:

Average: 71.3%. The official SIE passing score is 70%. So ChatGPT passed twice and failed once.

That’s not a comfortable margin. A human candidate scoring 71% on practice exams would not be ready to schedule the real test. And the failure pattern reveals more than the average score does.

Where did ChatGPT fail?

The misses concentrated in three areas.

1. Specific regulatory thresholds. Questions like “What is the minimum maintenance margin requirement under Reg T?” or “Within how many business days must a customer complaint be reported on Form U4?” ChatGPT often gave plausible-sounding but wrong numbers. In one run it confidently said maintenance margin was 50% (it’s 25% under FINRA, with Reg T’s 50% being the initial requirement, a different thing).

2. Prohibited-activities scenarios. Questions where you read a fact pattern and identify whether it’s churning, free-riding, selling away, or a permitted activity. The model can recognize the textbook definitions but struggles with the borderline cases that the SIE actually tests.

3. Newer or less common rules. Anything added or revised in the last 3 to 4 years showed up as either outdated or hedged (“this may have been updated, please verify”). Reg BI specifics, recent FINRA guidance on crypto-asset securities, and the May 2024 T+1 settlement transition were all weak spots.

Where it did well: capital markets fundamentals (issuers, dealers, market structure), product mechanics (how options work, how municipal bonds are taxed), and broad regulatory roles (SEC vs FINRA vs MSRB jurisdiction).

Why does it confidently get things wrong?

LLMs like ChatGPT generate text that looks like the right answer based on patterns in training data. They don’t have access to a structured database of FINRA rules. When the model has seen the right answer many times in its training corpus (e.g., “what is a stock?”), it gets it right. When the right answer is buried in obscure regulatory text and the wrong answers also appear plausibly in financial writing, it can fail.

The dangerous part isn’t that ChatGPT gets things wrong. Every study tool has an error rate. The dangerous part is that it gets things wrong with the same confident tone it uses when right. If you’re a beginner, you can’t tell the difference.

Did ChatGPT show its work?

Yes, and that’s where things got interesting. When asked to explain its reasoning, the model often produced clean, well-structured walkthroughs that referenced “FINRA Rule X” or “SEC Reg Y.” About 1 in 8 of those citations were either:

• The wrong rule number (real rule, but governs something different)
• A made-up rule number that sounds real
• A real rule cited for the wrong reason

These are textbook hallucinated citations. They’re harder to spot than wrong final answers because they look more authoritative.

Did better prompting help?

A bit. We tried two prompt variants:

Variant A (control): Just paste the question, ask for the answer.

Variant B (chain-of-thought): “Think step by step. Identify the topic area, the relevant rule or concept, then evaluate each answer choice.”

Variant C (role + verify): “You are a FINRA registered representative with 15 years of experience. Answer this SIE practice question. Before stating your final answer, verify the rule citation.”

Variant B improved the average to about 75%. Variant C improved to 76% but added a lot of latency. Neither got close to a comfortable pass margin (>80%).

The wins from prompting plateau quickly. The model is good at reasoning with the knowledge it has, but it can’t conjure facts it doesn’t reliably know.

Could ChatGPT pass with the FINRA outline in context?

We tried this too. We pasted the relevant section of the official FINRA SIE content outline directly into the conversation and asked the same questions.

Score jumped to 84%. That’s the kind of margin a real candidate should aim for.

But: this only works if the question is on a topic covered in the outline excerpt you pasted. The full FINRA outline plus the relevant rules and definitions runs to hundreds of pages, far more than fits in a single ChatGPT context window comfortably. And feeding the full corpus in for every question is exactly what a purpose-built study tool does, except a study tool also has the questions, the explanations, the spaced-repetition, and the human review that ChatGPT doesn’t.

In other words: when ChatGPT has access to the right reference material, it does well. So does any tool. The real question is whether it’s the right tool for daily SIE study, and the answer there is no.

What does this mean for your SIE prep?

A few practical takeaways.

Use ChatGPT as a tutor, not as a question writer or grader. Ask it to explain concepts you’re struggling with. “Explain the difference between Rule 144 and Rule 144A in plain English” is a good prompt. “Write me 20 SIE practice questions” is a bad one (the questions will look real but contain errors you can’t catch).

Verify everything. Treat ChatGPT explanations the way you’d treat a Reddit comment: useful pointer, not authority. Cross-check rule numbers and specific thresholds against FINRA, SEC, or a curated study tool.

Don’t grade your own practice with it. If you want to know whether your answer to a tricky question is right, the wrong place to ask is an LLM. The right place is an explanation written by someone who actually passed the exam and was reviewed by someone who teaches it.

Use it for vocabulary and intuition. ChatGPT is great at “Wait, what is a Direct Participation Program in 30 seconds?” or “Give me an analogy for how municipal bond taxation works.” That’s the lane where it adds value without adding risk.

Will ChatGPT pass the SIE in two years?

Probably yes, with improving margins. Newer model generations are scoring better on standardized tests across the board. By the time GPT-6 or Claude Sonnet 5 is out, an LLM with a long-context window and access to FINRA rule text will likely score in the 90s.

But three things will still be true:

1. You are taking the exam, not ChatGPT.
2. The skill the SIE tests is fast recall of specific facts, not reasoning ability. Even if a future LLM can score 95%, that doesn’t help you sit in a Pearson VUE testing center.
3. Hallucinated citations are an architectural problem with autoregressive language models, not a model-quality problem. Even much smarter models will still confabulate.

The right way to think about LLMs in SIE prep is not “Can it pass?” but “Where does it help me prepare?” The answer to the second question is more useful than the answer to the first.

The bottom line

ChatGPT can squeeze past the SIE passing threshold on a good day. That’s a fascinating data point and a terrible study strategy. The 30% it gets wrong includes exactly the kinds of regulatory specifics the exam loves to test, presented with the same confidence as the 70% it gets right. Use it as a concept tutor, not as a study companion. The hours you’d spend cross-checking its work are hours better spent on a curated question bank with verified explanations.