Deploying Fine-Tuned Models On-Premise for Law Firms: A Compliance Checklist

Deploying AI at a law firm is not just a technical project — it is a compliance exercise. Attorney-client privilege, professional conduct rules, and data protection regulations impose specific requirements that general-purpose deployment guides do not address.

This checklist covers every compliance consideration for on-premise deployment of fine-tuned AI models at law firms. Use it as a project management tool and as documentation evidence for the firm's compliance review.

Pre-Deployment: Data Handling

Training Data Governance

• Data sourcing documented. Record the source of all training data — which matters, which document types, which date range.
• Conflict check on training data. If the firm serves competing clients, ensure training data from Client A is never used in a model serving Client B. This mirrors the firm's existing conflict wall procedures.
• Client consent obtained. If training data derives from client matters, confirm the firm's engagement letters or data use policies permit use for AI training. Some firms include this in standard engagement terms; others need specific consent.
• Privilege review. Training data containing privileged communications should be flagged. While fine-tuning on privileged data deployed within the firm's own infrastructure does not waive privilege, the firm's ethics committee should be aware.
• Data minimisation applied. Only include data necessary for the specific AI task. Do not train on entire matter files when clause-level data suffices.
• Retention policy defined. How long is training data retained after fine-tuning? Who authorises deletion? Document this.

Data Preparation

• PII handling documented. If training data contains personal information (client names, addresses in contracts), document the handling protocol. For on-premise deployment, PII in training data is lower-risk since it stays within the firm, but should still be acknowledged.
• Data quality review completed. Lawyer reviewed a sample of training examples for accuracy and appropriateness.
• Data stored on firm infrastructure. Training data has not been uploaded to any cloud service, external platform, or personal device.

Access Controls

Model Access

• Role-based access implemented. Not every staff member should interact with the AI system. Define who can:
  • Submit queries to the model (typically: lawyers, paralegals)
  • View model outputs (same as above, plus matter-specific restrictions)
  • Modify workflows or configurations (IT administrators only)
  • Retrain or update the model (designated AI administrator)
• Authentication integrated. AI system authenticates against the firm's identity provider (Active Directory, Okta, etc.). No standalone credentials.
• Matter-based access controls. If the AI serves multiple practice groups or matters, ensure that query results are accessible only to authorised personnel for each matter.
• Client-specific model isolation. If per-client LoRA adapters are used, ensure adapter selection is tied to matter codes — a user working on Client A's matter cannot accidentally query Client B's model.

Infrastructure Access

• Physical access restricted. The GPU server is in a locked server room or secured data centre with logged access.
• SSH/remote access secured. Administrative access to the inference server uses SSH key authentication, not passwords.
• Network segmentation. The AI server is on a segregated network segment. It cannot reach the internet (or access is restricted to essential updates only).
• Firewall rules documented. Only authorised services (n8n, client portal) can communicate with the inference server.

Audit Logging

Inference Logging

• Every query logged. Record: timestamp, user identity, matter/client code, input text (or hash), model version, adapter used.
• Every response logged. Record: timestamp, output text (or hash), processing time, confidence indicators.
• Logs stored on firm infrastructure. Not sent to any external monitoring service.
• Log retention period defined. Align with the firm's document retention policy — typically 7+ years for matter-related records.
• Log integrity protected. Append-only logging or write-once storage to prevent tampering.
• Log access restricted. Only authorised personnel (IT, compliance) can access inference logs.

Model Change Logging

• Model version history maintained. Track which model version (base model + adapter) was active on each date.
• Training run records retained. For each fine-tuning run: training data version, configuration, date, operator, resulting model checksum.
• Deployment changes logged. When a model is updated, rolled back, or replaced, the change is recorded with justification.

Model Versioning

• Version numbering scheme defined. e.g., firm-contract-review-v1.2.3 (major.minor.patch)
• Previous versions archived. At least the last 3 model versions retained for rollback capability.
• Version deployed is documented. At any point, you can answer "which model version produced this output on this date?"
• Rollback procedure tested. If a new model version produces poor results, you can revert to the previous version within minutes.
• Model checksums recorded. SHA-256 hash of each model file for integrity verification.

Client Data Isolation

• Per-client adapters segregated. Each client's LoRA adapter stored in a separate directory with appropriate filesystem permissions.
• Cross-client inference prevented. Technical controls ensure a request from Matter A cannot load Client B's adapter.
• Client data deletion procedure. When a client engagement ends, procedure for deleting their training data, adapter, and inference logs is documented and tested.
• Data isolation audit. Periodically verify that no cross-client data contamination has occurred (review adapter training logs, check filesystem permissions).

Bar Association Requirements

ABA Model Rules

• Rule 1.1 (Competence) addressed. Lawyers using the AI system understand its capabilities and limitations. Training provided.
• Rule 1.4 (Communication) addressed. If AI is used on a client's matter, the firm's policy on disclosing AI use to clients is documented and followed.
• Rule 1.6 (Confidentiality) addressed. The on-premise deployment architecture ensures client information is not disclosed to any third party through AI processing.
• Rule 5.1/5.3 (Supervision) addressed. Partners/supervisors are responsible for ensuring AI outputs are reviewed by a qualified lawyer before being relied upon or delivered to clients.

State-Specific Requirements

• Applicable state bar opinions reviewed. Several state bars have issued opinions on AI use in legal practice. Review opinions for all jurisdictions where the firm is licensed.
• Disclosure requirements identified. Some jurisdictions require disclosure to courts when AI was used in preparing filings. Identify and comply with these requirements.
• Continuing obligations tracked. Bar association guidance on AI is evolving rapidly. Assign responsibility for monitoring and implementing new guidance.

Incident Response

• AI-specific incident types defined:
  • Model produces incorrect legal analysis that is relied upon
  • Unauthorised access to the AI system
  • Model produces output containing information from a different client's matter
  • Hardware failure causing AI system unavailability
  • Model produces biased or discriminatory output
• Response procedures documented for each incident type, including:
  • Immediate containment steps
  • Investigation procedure
  • Notification requirements (which partners, which clients, which regulators)
  • Remediation actions
  • Post-incident review
• Incident response team designated. Typically: managing partner, IT lead, compliance officer, AI administrator.
• Incident response drills conducted. At least annually, test the response procedure with a simulated incident.

Documentation for Compliance Review

Prepare a compliance package that the firm's ethics committee, managing partner, or external auditor can review:

Technical Documentation

• Architecture diagram. Shows all components, data flows, and network boundaries.
• Data flow diagram. Where client data enters the system, how it is processed, where outputs are delivered.
• Security controls summary. Authentication, encryption, access controls, network segmentation.
• Third-party component list. All software components with versions and licences (open-source models are licence-specific — verify compliance with Llama's or Mistral's licence terms).

Policy Documentation

• AI use policy. The firm's policy on when and how AI can be used in legal work.
• Acceptable use guidelines. What staff may and may not submit to the AI system.
• Training materials. Materials provided to staff on using the AI system responsibly.
• Vendor agreements. If using Ertas or other tools for fine-tuning, document the relationship and data handling practices.

Ongoing Compliance

• Quarterly review scheduled. Review model performance, audit logs, access patterns, and incident reports quarterly.
• Annual compliance assessment. Comprehensive review of the entire AI deployment against current bar association guidance and regulatory requirements.
• Update trigger defined. Events that trigger an out-of-cycle compliance review (new bar opinion, regulatory change, security incident, major model update).

---

Further Reading

• Data Sovereignty for AI Agencies — The broader data sovereignty framework for regulated industries
• Why Law Firms Won't Send Client Data to ChatGPT — The privilege and compliance case for on-premise AI in legal