character encoding - Inno Setup - Convert array of string to Unicode and back to ANSI

Question

I'm loading a Korean CP51949 (EUC-KR) encoded ANSI file into an array of strings (LoadStringsFromFile). My system and the intended end user systems do not have CP51949 set as a legacy non-Unicode encoding.

At the moment I have 2 problems with this:

1. Unless I run the application with Locale Emulator (which is just annoying, since the setup itself is in English only), the Korean text is displayed as gibberish.
2. Pos gives wrong results and StringChange fails completely unless I switch to String, do the thing, and then back to AnsiString.

So, I'm wondering if there's a way to convert the array to unicode, and then back before saving.

Answer 1

To convert a file encoded in a specific Ansi encoding to Unicode string (UTF-16 LE), use MultiByteToWideChar function:

function MultiByteToWideChar(
  CodePage: UINT; dwFlags: DWORD; const lpMultiByteStr: AnsiString;
    cchMultiByte: Integer; lpWideCharStr: string; cchWideChar: Integer): Integer;
  external 'MultiByteToWideChar@kernel32.dll stdcall';  

function LoadStringFromFileInCP(
  FileName: string; var S: string; CP: Integer): Boolean;
var
  Ansi: AnsiString;
  Len: Integer;
begin
  Result := LoadStringFromFile(FileName, Ansi);
  if Result then
  begin
    Len := MultiByteToWideChar(CP, 0, Ansi, Length(Ansi), S, 0);
    SetLength(S, Len);
    MultiByteToWideChar(CP, 0, Ansi, Length(Ansi), S, Len);
  end;
end;

function LoadStringsFromFileInCP(
  FileName: string; Strings: TStrings; CP: Integer): Boolean;
var
  S: string;
begin
  Result := LoadStringFromFileInCP(FileName, S, CP);
  if Result then Strings.Text := S;
end;

(Note that I'm using TStrings to store strings/lines collection instead of TArrayOfString, as TStrings is easier to work with)

---

To convert Unicode string back to Ansi, use WideCharToMultiByte function:

function WideCharToMultiByte(CodePage: UINT; dwFlags: DWORD;
  lpWideCharStr: string; cchWideChar: Integer; lpMultiByteStr: AnsiString;
  cchMultiByte: Integer; lpDefaultCharFake: Integer;
  lpUsedDefaultCharFake: Integer): Integer;
  external 'WideCharToMultiByte@kernel32.dll stdcall';

function SaveStringToFileInCP(FileName: string; S: string; CP: Integer): Boolean;
var
  Ansi: AnsiString;
  Len: Integer;
begin
  Len := WideCharToMultiByte(CP, 0, S, Length(S), Ansi, 0, 0, 0);
  SetLength(Ansi, Len);
  WideCharToMultiByte(CP, 0, S, Length(S), Ansi, Len, 0, 0);
  Result := SaveStringToFile(FileName, Ansi, False);
end;

function SaveStringsToFileInCP(
  FileName: string; Strings: TStrings; CP: Integer): Boolean;
begin
  Result := SaveStringToFileInCP(FileName, Strings.Text, CP);
end;

---

Use the functions like:

const
  CP_EUC_KOREAN = 51949;

var
  I: Integer;
  Strings: TStrings;
begin
  Strings := TStringList.Create;
  if LoadStringsFromFileInCP('korean.txt', Strings, CP_EUC_KOREAN) then
  begin
    for I := 0 to Strings.Count - 1 do
    begin
      MsgBox(Strings[I], mbInformation, MB_OK);
    end;
  end;

  SaveStringsToFileInCP('korean_out.txt', Strings, CP_EUC_KOREAN);
end;

---

Works correctly on my English-only system: