Turn Any Article into a High-Quality Question Bank with AI

“I have hundreds of journal papers and need solid questions for model training—fast.”
“Can my laptop handle it? Do I need a GPU?”
“Which provider is cheapest if I only want to test ten samples?”

If any of those questions sound familiar, this guide will give you exact, copy-paste answers.
We break the small open-source tool into plain steps, show working examples, and keep everything inside the original README—no external links, no fluff.


1. What the Tool Actually Does

Feed it any text, get N questions back.

  • Data source: Hugging Face datasets or local .jsonl, .json, .parquet.
  • Engine: 10+ large language model providers—OpenAI, Anthropic, Gemini, local Ollama, and more.
  • Tone control: 30+ built-in styles (academic, humorous, critical, kid-friendly…) chosen at random or forced by you.
  • Output: One JSON-L file per run. Each line carries the original text, the generated question, and full metadata for downstream cleaning.

2. Environment Setup in Three Commands

Step Command Why it matters
1. Optional virtual environment python3 -m venv .venv && source .venv/bin/activate Keeps dependencies tidy
2. Install packages pip install -r requirements.txt Only three libraries: aiohttp, datasets, tqdm
3. Export the API key export OPENROUTER_API_KEY=YOUR_KEY Use the provider you like; table below

No GPU is required—computation happens in the cloud, and your laptop only sends HTTP requests.


3. Run Your First 10 Lines in 30 Seconds

Copy-paste the block and hit Enter:

export OPENROUTER_API_KEY=YOUR_KEY
python3 src/main.py mkurman/hindawi-journals-2007-2023 \
  --provider openrouter \
  --model qwen/qwen3-235b-a22b-2507 \
  --output-dir ./data/demo \
  --start-index 0 \
  --end-index 10 \
  --num-questions 5 \
  --text-column text \
  --verbose

When it stops, ./data/demo contains a file like
questions_2025-08-18_14-22-03_hindawi-journals_openrouter_qwen.jsonl.


4. Every Flag Explained in One Table

Flag Example Value Purpose Common Pitfall
<dataset_or_path> mkurman/hindawi-journals-2007-2023 or ./myfile.jsonl Where to read text Local path must end with .jsonl, .json, or .parquet
--provider openrouter Which LLM gateway See provider list
--model qwen/qwen3-235b-a22b-2507 Exact model name Must match provider catalog
--output-dir ./results Where to drop JSONL Old files stay; new ones are time-stamped
--num-questions 3 Questions per text Too high may hit rate limits
--start-index / --end-index 0 / 100 Slice dataset Zero-based, end exclusive
--text-column text Column holding text Change if your file uses content
--style "formal and academic" Force tone One string or comma list
--no-style Neutral questions Overrides any style flag
--styles-file ./my_styles.txt Load from file One style per line, # comments
--num-workers 4 Parallel calls Combine with --sleep-between-requests
--sleep-between-requests 0.5 Seconds between calls Prevents HTTP 429
--max-items 1000 Global cap Alternative to --end-index

5. Provider & Key Cheat Sheet

Provider Environment Variable Notes
OpenAI OPENAI_API_KEY Works with official or proxy endpoints
Anthropic ANTHROPIC_API_KEY Claude family
OpenRouter OPENROUTER_API_KEY One key → many models
Groq GROQ_API_KEY Very fast inference
Together TOGETHER_API_KEY Strong open-source catalog
Cerebras CEREBRAS_API_KEY New chips, beta pricing
Gemini GEMINI_API_KEY Export as env var even if docs mention query param
Qwen (official) QWEN_API_KEY Tongyi Qianwen
Qwen (DeepInfra) QWEN_DEEPINFRA_API_KEY Alternative endpoint
Kimi (Moonshot) KIMI_API_KEY Chinese-first
Z.ai Z_AI_API_KEY Replace dots/ dashes with underscores
Featherless FEATHERLESS_API_KEY Budget friendly
Chutes CHUTES_API_KEY Same as above
Hugging Face HUGGINGFACE_API_KEY Needed only for private datasets
Ollama None Assumes http://localhost:11434

6. Three Ways to Feed the Tool

  1. Hugging Face Hub
    Use org/dataset, default split is train; add --dataset-split test if needed.

  2. Local JSONL
    One JSON object per line:

    {"text": "Large Language Models excel at generating diverse questions."}
    {"text": "Neural networks learn complex patterns from data."}
    
  3. Local Parquet
    Same --text-column rule applies.


7. Tone System Deep Dive

7.1 Default Behavior

No extra flags → tool picks one of 35+ styles per text.
Examples:

  • formal and academic
  • funny and entertaining
  • practical and application-focused
  • thought-provoking and philosophical

7.2 Single Tone

--style "formal and academic"

7.3 Random Pool

--style "casual and conversational,funny and humorous,concise and direct"

Each text gets one style from the pool.

7.4 File-Based Styles

Create my_styles.txt:

# Lines starting with # are ignored
formal and academic
creative and imaginative
simple and straightforward

Then:

--styles-file ./my_styles.txt

8. Output Anatomy

Successful record:

{
  "input": "What are the latest advances in text summarization?",
  "source_text": "Original paragraph...",
  "question_index": 1,
  "total_questions": 5,
  "metadata": { "original_item_index": 0, "text_column": "text" },
  "generation_settings": {
    "provider": "openrouter",
    "model": "qwen/qwen3-235b-a22b-2507",
    "style": "formal and academic",
    "num_questions_requested": 5,
    "num_questions_generated": 5,
    "max_tokens": 4096
  },
  "timestamp": "2025-08-17T12:34:56.789012"
}

Failure record:

{
  "error": "Rate limit exceeded",
  "source_text": "...",
  "metadata": { ... }
}

9. Quick-Fire FAQ

Q1: I only want 5 samples—shortest command?

--max-items 5
# or
--start-index 0 --end-index 5

Q2: How do I stay inside free tier limits?

--sleep-between-requests 1.0 --num-workers 1

Q3: Local Ollama workflow?

  1. Start Ollama and pull a model

    ollama run hf.co/lmstudio-community/Qwen3-4B-Instruct-2507-GGUF:Q4_K_M
    
  2. Run the script

    python3 src/main.py ./my.jsonl \
      --provider ollama \
      --model hf.co/lmstudio-community/Qwen3-4B-Instruct-2507-GGUF:Q4_K_M \
      --output-dir ./ollama_out
    

No key required.

Q4: Output files are huge—any rollover?
Timestamp is automatic; just schedule daily runs.

Q5: How custom can styles get?
Any short English phrase the model understands: --style "like a curious five-year-old".


10. Real-World Mini-Project

Turn 100 news articles into interview questions.

python3 src/main.py ./news.jsonl \
  --provider openrouter \
  --model qwen/qwen3-235b-a22b-2507 \
  --output-dir ./interview_q \
  --style "critical thinking for job interview" \
  --num-questions 3 \
  --max-items 100 \
  --sleep-between-requests 0.5

Open interview_q/questions_2025-08-18_xxx.jsonl, remove empty lines, and you’re done.


11. Troubleshooting Checklist

Error Message Root Cause Fix
ModuleNotFoundError: aiohttp Missing dependencies pip install -r requirements.txt
401 Unauthorized Wrong or unset key Re-export the correct variable
429 Rate limit Too many requests Increase --sleep-between-requests
Empty output Index out of range Check dataset length
Connection refused Ollama offline Ensure http://localhost:11434 responds

12. Next Steps

You now know how to:

  • Install the tool in under five minutes
  • Generate questions from any public or local dataset
  • Tune tone, rate limits, and concurrency
  • Read the output and fix common errors

What you can do next:

  1. Feed the question bank into a RAG pipeline.
  2. Maintain a company style file for customer-support or tech-interview scenarios.
  3. Load the JSONL into Excel or a database for human quality review.

Happy experimenting!