In mature Delphi-systems, Dataset-to-object mapping is rarely the clean „one field = one property“ case. In bespoke enterprise software you instead encounter alias columns from views, join results with duplicate field names, „empty“ values as 0 or ' ', typed fields that deliver VARCHAR today and INTEGER tomorrow, and columns that are simply missing depending on the search dialog. This is exactly where many mappers fail: either they become „magical“ (and therefore hard to debug), or they are so strict that a single optional field stops operation.
This source snippet shows a pragmatic mapper for Delphi that is intentionally not an ORM, but cleanly addresses the most important legacy edge cases: unambiguous field resolution, controlled conversion, null semantics, optional fields and traceable error messages. It is suitable for data access layers (DAL, i.e., a layer that encapsulates data access) or repository patterns — and integrates well with a BDE replacement with native connectivity (Delphi data access library for many DBs).
Why standard mapping fails on legacy structures
A few typical operational causes that are rare in a „clean“ redesign:
- Ambiguous field names: a join returns
IDfrom multiple tables; in the dataset it may appear asID,ID_1or be renamed via an SQL alias. - Semantic nulls:
0means „unknown“,'1899-12-30'is „no date“,' 'is „not populated“. - Varying types: a view does not cast; the driver returns
ftWideStringinstead offtInteger. Variant conversion becomes a source of errors. - Optional columns: a search dialog uses different SELECT lists depending on the filter. Code, however, expects fields to be present „always“.
- Debuggability: when mapping disappears into RTTI, debugging customer data is difficult (which field, which value, which type?).
Approach: mapping plan instead of convention, with controlled conversion
The core is a mapping plan: a list of rules „Property X comes from field A or B, is optional/required, uses converter Y“. This keeps the mapping declarative, but not „invisible“ like many ORM mechanisms. In addition, the mapper can throw a meaningful exception per field, including field name, data type and raw value.
Important: we deliberately map from TDataSet, not from a concrete BDE-Ablosung mit nativer Anbindung class. That preserves compatibility with TFDQuery, TClientDataSet or third-party components.
Source snippet: debuggable dataset-to-object mapping for legacy columns
The code implements:
- Field resolution via a priority list (aliases/fallbacks)
- Required/optional handling
- Null semantics via converters (e.g.
0 => Null) - Stable error messages with context
- A debug hook to reproduce mapping issues in tests or in support cases
unit Legacy.DatasetMapper;
interface
uses
System.SysUtils, System.Variants, System.Generics.Collections, Data.DB;
type
EDataMappingError = class(Exception)
private
FFieldNames: string;
FTarget: string;
FDataType: string;
FRawValue: string;
public
constructor Create(const ATarget, AFieldNames, ADataType, ARawValue, AMsg: string);
property Target: string read FTarget;
property FieldNames: string read FFieldNames;
property DataType: string read FDataType;
property RawValue: string read FRawValue;
end;
TMapRequired = (mrOptional, mrRequired);
TMapDebugEvent = reference to procedure(
const TargetMember: string;
const SourceField: string;
const SourceType: TFieldType;
const SourceValue: Variant);
// Converter receives Variant and returns Variant (e.g. Null, Integer, String, TDateTime as Double)
TFieldConverter = reference to function(const V: Variant): Variant;
TFieldSpec = record
TargetMember: string;
SourceCandidates: TArray<string>;
Required: TMapRequired;
Converter: TFieldConverter;
class function Create(const ATarget: string; const ACandidates: array of string;
ARequired: TMapRequired; const AConverter: TFieldConverter): TFieldSpec; static;
end;
TLegacyDatasetMapper = class
private
FOnDebug: TMapDebugEvent;
function FindFieldByCandidates(DS: TDataSet; const Candidates: TArray<string>): TField;
function FieldTypeToString(FT: TFieldType): string;
function VariantToDiag(const V: Variant): string;
public
property OnDebug: TMapDebugEvent read FOnDebug write FOnDebug;
// MapOne: calls setter for each spec. No RTTI: explicit assignment is easier to debug.
procedure MapOne(DS: TDataSet; const Specs: TArray<TFieldSpec>;
const Assign: TProc<string, Variant>);
end;
// Helper converters
function C_TrimToNull: TFieldConverter;
function C_ZeroToNull: TFieldConverter;
function C_StrictInt: TFieldConverter;
function C_DateFromStringOrNull: TFieldConverter;
implementation
{ EDataMappingError }
constructor EDataMappingError.Create(const ATarget, AFieldNames, ADataType, ARawValue, AMsg: string);
begin
inherited Create(AMsg);
FTarget := ATarget;
FFieldNames := AFieldNames;
FDataType := ADataType;
FRawValue := ARawValue;
end;
{ TFieldSpec }
class function TFieldSpec.Create(const ATarget: string; const ACandidates: array of string;
ARequired: TMapRequired; const AConverter: TFieldConverter): TFieldSpec;
var
I: Integer;
begin
Result.TargetMember := ATarget;
SetLength(Result.SourceCandidates, Length(ACandidates));
for I := 0 to High(ACandidates) do
Result.SourceCandidates[I] := ACandidates[I];
Result.Required := ARequired;
Result.Converter := AConverter;
end;
{ TLegacyDatasetMapper }
function TLegacyDatasetMapper.FieldTypeToString(FT: TFieldType): string;
begin
Result := GetEnumName(TypeInfo(TFieldType), Ord(FT));
end;
function TLegacyDatasetMapper.VariantToDiag(const V: Variant): string;
begin
if VarIsNull(V) then Exit('NULL');
if VarIsEmpty(V) then Exit('EMPTY');
try
Result := VarToStr(V);
except
Result := '<unprintable variant>';
end;
end;
function TLegacyDatasetMapper.FindFieldByCandidates(DS: TDataSet; const Candidates: TArray<string>): TField;
var
Name: string;
begin
Result := nil;
for Name in Candidates do
begin
// FindField instead of FieldByName: optional, no exception
Result := DS.FindField(Name);
if Result <> nil then
Exit;
end;
end;
procedure TLegacyDatasetMapper.MapOne(DS: TDataSet; const Specs: TArray<TFieldSpec>;
const Assign: TProc<string, Variant>);
var
Spec: TFieldSpec;
F: TField;
Raw, Val: Variant;
CandidatesJoined: string;
I: Integer;
FT: string;
begin
if (DS = nil) then
raise EArgumentNilException.Create('DS');
if not DS.Active then
raise EInvalidOperation.Create('Dataset is not active.');
for Spec in Specs do
begin
F := FindFieldByCandidates(DS, Spec.SourceCandidates);
if (F = nil) then
begin
if Spec.Required = mrRequired then
begin
CandidatesJoined := '';
for I := 0 to High(Spec.SourceCandidates) do
begin
if I > 0 then CandidatesJoined := CandidatesJoined + ', ';
CandidatesJoined := CandidatesJoined + Spec.SourceCandidates[I];
end;
raise EDataMappingError.Create(
Spec.TargetMember,
CandidatesJoined,
'n/a',
'n/a',
Format('Mapping error: Required field for %s not found. Candidates: [%s]',
[Spec.TargetMember, CandidatesJoined]));
end
else
Continue; // optional: simply skip
end;
Raw := F.Value; // Variant; accounts for Null
if Assigned(FOnDebug) then
FOnDebug(Spec.TargetMember, F.FieldName, F.DataType, Raw);
try
if Assigned(Spec.Converter) then
Val := Spec.Converter(Raw)
else
Val := Raw;
// Required: Null after converter is an error (more common than you might think)
if (Spec.Required = mrRequired) and VarIsNull(Val) then
begin
FT := FieldTypeToString(F.DataType);
raise EDataMappingError.Create(
Spec.TargetMember,
F.FieldName,
FT,
VariantToDiag(Raw),
Format('Mapping error: %s is required, but value is NULL after conversion. Field %s (%s), raw value=%s',
[Spec.TargetMember, F.FieldName, FT, VariantToDiag(Raw)]));
end;
Assign(Spec.TargetMember, Val);
except
on E: EDataMappingError do
raise;
on E: Exception do
begin
FT := FieldTypeToString(F.DataType);
raise EDataMappingError.Create(
Spec.TargetMember,
F.FieldName,
FT,
VariantToDiag(Raw),
Format('Mapping error for %s from field %s (%s), raw value=%s: %s',
[Spec.TargetMember, F.FieldName, FT, VariantToDiag(Raw), E.Message]));
end;
end;
end;
end;
{ Converters }
function C_TrimToNull: TFieldConverter;
begin
Result := function(const V: Variant): Variant
var
S: string;
begin
if VarIsNull(V) or VarIsEmpty(V) then Exit(Null);
S := Trim(VarToStr(V));
if S = '' then
Result := Null
else
Result := S;
end;
end;
function C_ZeroToNull: TFieldConverter;
begin
Result := function(const V: Variant): Variant
var
N: Int64;
begin
if VarIsNull(V) or VarIsEmpty(V) then Exit(Null);
// also tolerates '0' as a string
N := StrToInt64(Trim(VarToStr(V)));
if N = 0 then
Result := Null
else
Result := N;
end;
end;
function C_StrictInt: TFieldConverter;
begin
Result := function(const V: Variant): Variant
begin
if VarIsNull(V) or VarIsEmpty(V) then Exit(Null);
Result := StrToInt(Trim(VarToStr(V)));
end;
end;
function C_DateFromStringOrNull: TFieldConverter;
begin
Result := function(const V: Variant): Variant
var
S: string;
D: TDateTime;
begin
if VarIsNull(V) or VarIsEmpty(V) then Exit(Null);
S := Trim(VarToStr(V));
if (S = '') or (S = '1899-12-30') then Exit(Null);
// Intentionally strict: no "Try" that swallows data quality.
// Format may vary depending on legacy; optionally parameterize here via TFormatSettings.
D := ISO8601ToDate(S, False);
Result := D;
end;
end;
end.
How to use the mapper in practice (without RTTI, yet elegantly)
The mapper calls an Assign(TargetMember, Value) callback function. This keeps the assignment explicit (and therefore easy to debug) and avoids RTTI accesses on the hot path. In practice you build a small ‚assigner‘ per object/DTO (Data Transfer Object, i.e. a transport object for data).
type
TCustomer = class
public
Id: Integer;
ExternalNo: string;
DisplayName: string;
BirthDate: TDateTime; // optional in Legacy
end;
function MapCustomer(DS: TDataSet; Mapper: TLegacyDatasetMapper): TCustomer;
var
C: TCustomer;
Specs: TArray<TFieldSpec>;
begin
C := TCustomer.Create;
try
Specs := [
TFieldSpec.Create('Id', ['CUSTOMER_ID', 'ID', 'C_ID'], mrRequired, C_StrictInt),
TFieldSpec.Create('ExternalNo', ['EXT_NO', 'CUSTOMERNO'], mrOptional, C_TrimToNull),
TFieldSpec.Create('DisplayName', ['NAME', 'DISPLAYNAME', 'C_NAME'], mrRequired, C_TrimToNull),
TFieldSpec.Create('BirthDate', ['BIRTHDATE', 'DOB'], mrOptional, C_DateFromStringOrNull)
];
Mapper.MapOne(DS, Specs,
procedure(const Target: string; const V: Variant)
begin
if Target = 'Id' then
C.Id := V
else if Target = 'ExternalNo' then
C.ExternalNo := VarToStrDef(V, '')
else if Target = 'DisplayName' then
C.DisplayName := VarToStr(V)
else if Target = 'BirthDate' then
begin
if VarIsNull(V) then
C.BirthDate := 0
else
C.BirthDate := V;
end
else
raise EInvalidOperation.Create('Unknown TargetMember: ' + Target);
end);
Result := C;
except
C.Free;
raise;
end;
end;Purpose: The mapping is described centrally in one place (Specs), but the assignment remains explicit. In legacy situations this is usually the better trade-off decision than a fully automatic RTTI mapping, because you immediately see which property depends on which field names.
Preconditions: The approach expects an active DataSet and a current record position. For batch imports iterate externally over while not DS.Eof do and call MapCustomer per row.
Pitfalls: Watch VarToStr with BLOBs or memo fields; you should use custom converters there. And: ‚Required‘ means here after the converter. If C_TrimToNull sets a required field to null, that is intentional — data quality must then be resolved at the source or in the process.
Variants: Instead of string targets you can use an enum to avoid typos. Alternatively, you can store the Assign function per spec as a TProc<Variant>, then the target string disappears completely (slightly more boilerplate, but even fewer potential errors).
Context in architecture: DAL/Repository, logging and operations
In a layered architecture (typical: UI – Business – Data Access) this mapping belongs in the data access layer or in a repository. It is important that the DataSet is not „passed through“: objects/DTOs are the more stable interface, especially if you later retrofit REST APIs or offload parts to C# Services.
For operations and support the debug hook OnDebug is useful. You can use it in tests or in reproducible support cases to log which fields were actually mapped. In production systems this should be targeted and switchable; otherwise logging becomes too costly or too data-intensive.
Using the debug hook effectively
- Unit tests: Verify that a given SQL statement actually returns all required fields.
- Diagnosis: For customer issues you immediately see ‚field was not present‘ vs. ‚value could not be converted‘.
- Migration phases: When changing views/column names you can maintain candidate lists in parallel until everything has been migrated.
When this approach breaks down (and what to do instead)
The dataset-to-object mapping shown is robust when the data source is unstable and you still require deterministic behavior. It typically breaks down in two situations:
- Very large volumes (e.g. mass export): variant conversion and searching by field name can become noticeable. In that case a precomputed field-index cache per SQL pays off (e.g.
FieldByNameonce per dataset, not per row). - Very many DTO types: If you end up writing hundreds of mappers, boilerplate becomes an issue. Then an RTTI-based approach with attributes can make sense — but only if you strictly control debug outputs and converters.
A good middle ground is: field resolution and conversion as shown here (explicit, tolerant where necessary), but with generated code (e.g. via internal templates) instead of ‚handwritten‘ code.
Conclusion: Stability through explicit rules — with clear limits
With legacy datasets that include aliases, optional columns and historical null semantics, dataset-to-object mapping is most successful when it remains explicit and diagnosable. The mapping plan of candidate lists, required/optional flags and converters achieves exactly that: you can stabilize legacy artifacts step by step without introducing an ORM immediately or ’normalizing‘ the database all at once.
The limits are reached with extreme performance requirements and with very many types — then you need caching or automated code generation. For typical business software with evolved processes, however, the approach is a reliable lever to decouple data access and domain models and make them maintainable again.
If you need a second opinion or a sound target architecture for a concrete legacy mapping (FireDAC, views, join proliferation, null semantics), the next step is usually a short analysis with reproducible examples. Contact:
In the technical context, Delphi dataset mapping and legacy Delphi also play an important role when integrations, data flows and further development must work together cleanly.
Discuss a project or modernization initiative with Net-Base.