Text/sketch-to-CAD modelling & manufacturing drawing automation copilot

Mechanical engineers spend a large share of design time on repetitive CAD work: drawing and constraining sketches, building standard features and producing dimensioned drawings. Copilots now generate editable, feature-based CAD from a prompt or sketch, auto-constrain geometry and derive first-draft drawings from 3D models. The engineer stays the author: the AI proposes, the human checks fit, form, function and manufacturability.

Problem & value

• Today: parts are modelled by hand in SolidWorks, NX, Creo, Inventor, Fusion or Onshape. Drawings (views, dimensions, tolerances, notes, title blocks) are produced largely manually and are a known bottleneck between design and manufacturing. Variants of existing parts are rebuilt rather than reused.
• AI value: faster first drafts of simple and medium-complexity parts, automatic sketch constraining, natural-language command access in complex CAD UIs, template-driven drawing generation, and scripted CAD that can be version-controlled and reviewed like code.
• Evidence level: vendors report productivity gains in their own materials; independent, quantified field studies are still scarce, so measure on your own part portfolio.
• Not the same as: physics-driven topology optimisation (SCI-006), mesh/game assets (GEN-008), visual ideation (GEN-014) or reading existing drawings (DOC-013).

Variants

• Text-to-part: "bracket 80x40x5 mm with four M6 holes on a 60x20 pattern" → parametric part.
• Sketch/image-to-CAD: photo or hand sketch → constrained sketch → extrusions/revolves.
• Command copilot: natural-language access to CAD functions, help and macros inside the CAD tool.
• Auto-constraint and sketch repair: AI proposes dimensions and geometric constraints.
• Drawing automation: 3D model → views, sections, dimensions, hole tables, BOM, title block per company standard (ISO 128/ASME Y14.5 conventions).
• Design-table/variant generation: configurators for families of parts (cross-link KNOW-020).
• Private/maker use: 3D-printing parts via OpenSCAD/CadQuery generation with an LLM.

How it works

1. prompt / sketch / reference image
2. intent parsing (LLM/VLM: dimensions, features, units, material, process)
3. CAD program generation (feature sequence or script: sketch
4. extrude
5. fillet
6. pattern)
7. kernel execution (B-rep kernel builds the solid; errors fed back to the model)
8. validation (geometry checks, constraints solved, DFM rules, unit tests on dimensions)
9. human review in the CAD tool (edit parameters, feature tree stays editable)
10. drawing generation (view layout, dimension placement, tolerance & note templates)
11. PDM/PLM check-in with release workflow

• Core idea: the model emits a program (CAD script or command sequence) rather than a mesh, so the result remains parametric and the kernel guarantees valid solids. Execution errors are looped back (generate → run → repair).
• Constraint solving stays deterministic (CAD solver); AI suggests which constraints/dimensions to add.
• Drawings: rule-based placement plus AI for choosing views, datum schemes and tolerance proposals; GD&T and fits are always engineer-approved.
• Human-in-the-loop: review of geometry against requirement, approval of tolerances, release in PLM.

Data preparation

Needed are existing parametric CAD models with feature trees, parts catalogues, drawing standards, title-block templates, tolerance rules and design guidelines. These sit in the CAD/PDM or PLM system, on file shares, in old 2D drawings (PDF or paper) and in the heads of senior designers. Typical gaps are inconsistent naming and feature styles, models without parameters or with broken histories, drawings that deviate from the standard, duplicates and old revisions, and unwritten design rules. Access rights in PDM are often too broad for confidential customer geometry.

Preparation steps: 1. Define the standard parts, naming and template set that the copilot must follow (PREP-015, PREP-019). 2. Select clean example models and drawings as gold standards for prompts or fine-tuning (PREP-016). 3. Write design and drawing rules in short, retrievable form (PREP-013, PREP-014). 4. Check PDM permissions and classify confidential geometry before enabling cloud features (PREP-023, PREP-024). 5. Digitise relevant paper or PDF drawings where history matters (PREP-006, DOC-014).

Expect around 20 to 30 percent of the effort in data and rules preparation. Path A needs the least, B and C need curated examples and a test set of typical parts with known-good dimensions.

Implementation paths

Which path for whom

• SMB: Path A, because the CAD suite already licensed by the firm increasingly ships copilot and drawing-automation features, which avoids integration work.
• Enterprise: Path B, because company standards, parts catalogues, title blocks and PLM release workflows have to be built in through CAD APIs and no product covers them fully.
• Developer: Path C, because CadQuery, build123d or FreeCAD with an open-weight coder LLM and a sandbox give full control and run at HW2/HW3.

Path A — Buy (recommended for most SMBs)

• In-CAD copilots: Siemens NX / Designcenter with AI assistance (Siemens, DE) and Solid Edge; Autodesk Fusion with AI auto-constrain and the Autodesk Assistant; Dassault Systèmes (FR, EU) SOLIDWORKS and 3DEXPERIENCE AI features; PTC Creo/Onshape AI features; Zoo (text-to-CAD, KittyCAD).
• Drawing automation: drawing modules and add-ins of the major CAD suites, plus specialist add-in tools — verify support for your standard, language and title block.
• Choose A when: you already license one major CAD system and the vendor ships the feature. Check EU data residency and whether prompts/geometry are used for vendor model training.

Path B — Build on cloud APIs

• Text-to-CAD APIs: Zoo's Text-to-CAD API produces KCL and B-rep exports (STEP, glTF, etc.).
• LLM-as-CAD-programmer: frontier LLMs (Claude, GPT, Gemini; Mistral for EU hosting) generate CadQuery, build123d, OpenSCAD or FreeCAD Python; run in a sandbox with a geometry kernel, return renders and measurements to a VLM/LLM for self-critique.
• Integration: CAD vendor APIs (SOLIDWORKS API, Onshape FeatureScript/REST, Fusion API, NX Open) to create features in the native feature tree; MCP servers for CAD tools are appearing.
• Choose B when: you need company-specific parts catalogues, naming or standards baked in.

Path C — Self-host (open source)

• Kernels/libraries: CadQuery, build123d (OpenCascade), OpenSCAD, FreeCAD, Truck and CAD-focused open KCL/Zoo components.
• Models: open-weight coder/reasoning LLMs (Qwen3-Coder class, DeepSeek, gpt-oss) and open VLMs (Qwen-VL); research models such as Text2CAD and CAD-Coder have public papers/code.
• Hardware: HW2 for ≤30B quantised coders; HW3 for larger models and sketch-to-CAD VLMs.
• Effort: a sandboxed execution service, test harness and CAD-viewer review UI; ongoing ops.

Path D — Custom training

Worth it when you have a large library of your own parametric models (feature trees with history) in a consistent style, e.g. a fixed product family. Fine-tune a code LLM on (description, CAD script) pairs, use kernel execution as reward (RL with verifiable rewards). Public research datasets (DeepCAD, ABC, Fusion 360 Gallery, Text2CAD) are mostly simple sketch-extrude parts and do not match industrial assemblies; expect to curate several thousand in-house examples.

State of the art (2026)

• Programmatic CAD generation (LLM writes CadQuery/KCL) is the dominant approach; research systems (Text2CAD, CAD-Coder, CADFusion-style visual feedback loops) improve with executable-feedback and RL, but benchmarks centre on sketch-extrude parts [4][5].
• Vendor copilots are moving from help/search chat to acting on the model: auto-constraint, command execution, parametric feature creation and drawing drafts.
• Limits and failure modes: hallucinated or wrong dimensions, missing design intent (references, mating features), fragile feature trees that break on edit, weak performance on assemblies, freeform surfaces and complex multi-body parts, lack of tolerance/fit reasoning, and unreliable dimension placement on dense drawings. Geometric validity does not mean functional correctness.
• Practical rule: treat output as a draft for simple/medium parts; always verify against the spec.

Shared resources & combinations

• Same stack: llm-code + sandboxed execution as in other code-agent cases; VLM as in DOC-013.
• Upstream: SCI-013 (requirements → design parameters), SCI-006 (topology-optimised shapes re-modelled as clean parametric CAD), GEN-014 (concept images → sketch-to-CAD).
• Downstream: SCI-023 (CAM/NC programming), PRED-032 (should-cost from CAD), SCI-005 (simulation setup and meshing).
• Reuse: KNOW-020 (find existing similar parts before generating new ones); DOC-013 for the reverse direction (read drawings into data).

Effort, cost & KPIs

• Path A: days to weeks; add-on licences or per-user AI subscriptions.
• Path B: 2–4 months for a company-specific generator with sandbox, validators and CAD integration.
• Path C: 3–6 months; GPU server, ops and maintenance of the harness.
• Path D: months plus data curation.
• KPIs: time to first valid model, share of AI-generated features accepted unchanged, geometric/ dimensional error rate vs. spec, drawing preparation time, drawing review rework rate, part reuse rate.

People & change

Begin with management and the head of design/engineering, who set goals and decide how far drawing automation may go. Involve the design engineers from the start: they review geometry, approve tolerances and release in PLM, and they know the unwritten design rules. IT and IT security cover licences, data residency, sandboxing and vendor terms on training with prompts and geometry (GUIDE-010, GUIDE-013). Legal/compliance checks customer confidentiality and IP.

Works council: depends. In-CAD copilots usually log prompts and usage, and the BAG treats objective suitability for monitoring as sufficient for co-determination under §87(1) no. 6 BetrVG. The introduction also changes work processes, so inform the works council under §90(1) no. 3 and agree on logging limits in a short works agreement (GUIDE-004). Personal data are limited to user logs; the DPO should check vendor telemetry and the AVV (GUIDE-005).

AI literacy: train designers on prompting, checking generated geometry and why tolerances and GD&T always need human approval (GUIDE-006). Choose a pilot team with one product family and measure modelling and drawing time. Typical concern: de-skilling or loss of design creativity. Address it by positioning the copilot as a drafting aid, keeping review mandatory and gathering feedback in regular retrospectives (GUIDE-008, GUIDE-009).

Risks, regulation & compliance

• EU AI Act: minimal risk — design tooling is not an Annex III use; AI literacy (Art. 4) applies. If the CAD output is part of a safety component in a regulated product, the product law (Machinery Regulation (EU) 2023/1230, medical devices, etc.) applies to the product, not the tool; the engineer's verification duty is unchanged. No Art. 5 prohibited practice is involved.
• GDPR: normally no personal data; user prompts and author metadata can be personal data of staff.
• IP and trade secrets: CAD geometry is crown-jewel IP; check vendor training-data use, EU hosting, and export-control relevance (dual-use drawings). Cloud APIs need contractual protection.
• Liability and quality: AI-made geometry or tolerances must pass normal design review, FMEA (SCI-025) and release workflow; log which features were AI-generated for traceability.
• Licensing: training-data licences of public CAD datasets vary; check before commercial fine-tuning.

Getting started

1. Select 20 representative parts (simple brackets to medium housings) and 10 drawings as a benchmark.
2. Try the copilot in your existing CAD vendor tool first; record accepted vs. corrected output.
3. Prototype text-to-CadQuery with a sandbox and a render-and-critique loop on your benchmark.
4. Encode drawing standards (view rules, title block, tolerance defaults) as templates before adding AI.
5. Define review gates in PLM: AI-generated features flagged, engineer approval required.

Sources

1. Zoo (KittyCAD) — Text-to-CAD documentation — https://zoo.dev/docs
2. Autodesk — Fusion product information — https://www.autodesk.com/products/fusion-360/overview
3. Siemens — NX product page — https://plm.sw.siemens.com/en-US/nx/
4. Khan et al. — Text2CAD (NeurIPS 2024) — https://arxiv.org/abs/2409.17106
5. Doris et al. — CAD-Coder — https://arxiv.org/abs/2505.19713
6. CadQuery project — https://github.com/CadQuery/cadquery
7. build123d — https://github.com/gumyr/build123d
8. Regulation (EU) 2024/1689 (AI Act) — https://eur-lex.europa.eu/eli/reg/2024/1689/oj
9. Regulation (EU) 2023/1230 (Machinery) — https://eur-lex.europa.eu/eli/reg/2023/1230/oj