Unguarded local LLMs help scammers: KZ Scam Shield test

A developer tested local Llama 3.1 8B, Qwen 2.5 7B and DeepSeek-R1 8B on the KZ Scam Shield benchmark of 12 fraud schemes in Russian, Kazakh and mixed language. Unguarded models warned about fraud in only 2–10 of 42 cases, while a rule layer called MoneyGate blocked all 42 requests.

• Llama 3.1 8B unguarded warned about fraud in 10 of 42 cases
• Qwen 2.5 7B and DeepSeek-R1 8B warned in 2 of 42, near zero in Kazakh
• MoneyGate blocked all 42 scam requests across all three models
• Tests ran offline on a laptop with an 8 GB RTX 5050; a 12B model did not fit