ConvertGlobalStringTextEncoding

ConvertGlobalStringTextEncoding [flags ] originalTextEncoding, newTextEncoding , [string , string, ...]

The ConvertGlobalStringTextEncoding operation converts the contents of the specified global string variables or the contents of all global string variables in the data folder specified with the /DF flag from one text encoding to another.

The ConvertGlobalStringTextEncoding operation was added in Igor Pro 7.00.

By default, the contents of global string variables containing control characters (codes less than 32) other than carriage return (13), linefeed (10), and tab (9) are not converted. To convert the contents of global string variables containing control characters, you must include the /SKIP=0 flag.

Also by default, in Igor Pro 9.00 or later, when converting to UTF-8, strings that are already valid as UTF-8 are skipped.

In Igor Pro 7 or later, the contents of all global string variables are assumed to be encoded as UTF-8. You should convert the text encoding of a global string variable if, for example, you have global string variables in experiments created prior to Igor 7 and they contain non-ASCII characters. For those strings to be displayed and interpreted correctly, their contents need to be converted to UTF-8.

Most users will have no need to worry about the text encoding of Igor global string variables since most global string variables do not contain non-ASCII text. You should not use this operation unless you have a thorough understanding of text encoding issues or are instructed to use it by someone who has a thorough understanding.

See String Variable Text Encodings for essential background information.

ConvertGlobalStringTextEncoding can work on a list of specific global string variables or on all of the global string variables in a data folder (/DF flag). When working on a data folder, it can work on just the data folder itself or recursively on sub-data folders as well.

Conversion does not change the characters that make up text - it merely changes the numeric codes used to represent those characters.

Parameters

originalTextEncoding specifies the original (current) text encoding used by the specified global string variables. See Text Encoding Names and Codes for a list of codes.

newTextEncoding specifies the output text encoding. See Text Encoding Names and Codes for a list of codes.

string , string , ... is a list of the targeted global string variables. Only the name of string variables, not a path specification plus the name, is allowed. Therefore, the string variables must be in the current data folder. The list is optional and must be omitted if you use the /DF flag. Using both the /DF flag and a list of string variables is treated as an error. Use of local string variables in this list is also an error. Use ConvertTextEncoding to convert local string variables.

Flags

/CONV={errorMode [, diagnosticsFlags ]}
	errorMode determines how ConvertGlobalStringTextEncoding behaves if the conversion cannot be done because the text cannot be mapped to the specified text encoding. This will occur if the text contains characters that cannot be represented in the specified text encoding or if the value of originalTextEncoding is wrong.
	errorMode takes one of these values:
	1: Generate error. ConvertGlobalStringTextEncoding returns an error to Igor. 2: Use a substitute character for any unmappable characters. The substitute character for Unicode is the Unicode replacement character, � (U+FFFD). For most non-Unicode text encodings it is either control-Z or a question mark. 3: Skip unmappable input characters. Any unmappable characters will be missing in the output. 4: Use escape sequences representing any unmappable characters or invalid source text. If the source text is valid in the source text encoding but cannot be represented in the destination text encoding, unmappable characters are replaced with \uXXXX where XXXX specifies the UTF-16 code point of the unmappable character in hexadecimal. If the conversion cannot be done because the source text is not valid in the source text encoding, invalid bytes are replaced with \xXX where XX specifies the value of the invalid byte in hexadecimal.
	diagnosticsFlags is an optional bitwise parameter defined as follows:
	Bit 0: Emit diagnostic message if text conversion succeeds. Bit 1: Emit diagnostic message if text conversion fails. Bit 2: Emit diagnostic message if text conversion is skipped. Bit 3: Emit summary diagnostic message.
	All other bits are reserved for future use.
	See Setting Bit Parameters for details about bit settings.
	diagnosticsFlags defaults to 2 (bit 1 set) if the /DF flag is not present and to 10 (bits 1 and 3 set) if the /DF flag is present.
	Text conversion may be skipped because the global string variable's contents are detected as being binary or because newTextEncoding is the same as originalTextEncoding.
/DF={dfr, recurse, excludedDFR }
	dfr is a reference to a data folder. ConvertGlobalStringTextEncoding operates on all global string variables in the specified data folder. If dfr is null ($"") ConvertGlobalStringTextEncoding acts as if /DF was omitted. If the /DF flag is used, the optional string list parameter must not be used.
	If recurse is 1, ConvertGlobalStringTextEncoding works recursively on all sub-data folders. Otherwise it affects only the data folder referenced by dfr.
	excludedDFR is an optional reference to a data folder to be skipped by ConvertGlobalStringTextEncoding. For example, this command converts the string data for all global string variables in all data folders except for root:Packages and its sub-data folders:
	ConvertGlobalStringTextEncoding /DF={root:,1,root:Packages} 4, 1
	If excludedDFR is null ($"") ConvertGlobalStringTextEncoding acts as if excludedDFR was omitted and no data folders are excluded.
/SKIP=skip	Skips conversion of a string variable as follows:
	skip=0: Do not skip conversion. skip=1: Skip conversion of strings containing binary data. skip=2: When converting to UTF-8, skip conversion of strings that are already valid as UTF-8. This includes strings that contain only ASCII characters. skip=3: Skips both strings containing binary data and, when converting to UTF-8, strings that are already valid as UTF-8. This is the default behavior if /SKIP is omitted. /SKIP=2 and /SKIP=3 require Igor Pro 9.00 or later.
/Z[=z ]	Prevents procedure execution from aborting if ConvertGlobalStringTextEncoding generates an error. Use /Z or the equivalent, /Z=1, if you want to handle errors in your procedures rather than having execution abort.
	/Z does not suppress invalid parameter errors. It suppresses only errors in doing text encoding conversion.

Details

For general background information on text encodings, see Text Encodings.

For background information on string variable text encodings, see String Variable Text Encodings.

Using ConvertGlobalStringTextEncoding

ConvertGlobalStringTextEncoding is used to change the numeric codes representing the text - i.e., to convert the content to a different text encoding. Its main use is to convert Igor Pro 6 global string variables from whatever text encoding they use to UTF-8. UTF-8 is a form of Unicode which is more modern but is not backward compatible with Igor Pro 6. Igor Pro 7 and later assume that string variables are encoded as UTF-8.

For example, if you have global string variables from Igor Pro 6 that are encoded in Japanese (Shift JIS), you should convert them to UTF-8. Otherwise you will get errors or garbled text when you print or display the strings. This also applies to western text containing non-ASCII characters encoded as MacRoman or Windows-1252.

If you have an Igor 6 experiment containing non-ASCII string variables encoded as, for example, MacRoman, and you add some non-ASCII string variables in Igor 7 or later, you now have a mix of MacRoman and UTF-8 non-ASCII string variables. In Igor 9 or later, when you are converting to UTF-8, by default ConvertGlobalStringTextEncoding skips conversion of strings that are already valid as UTF-8. This allows you to convert the MacRoman string variables to UTF-8 without messing up the string variables that are already UTF-8.

Output Variables

The ConvertGlobalStringTextEncoding operation returns information in the following variables:

V_numConversionsSucceeded	Set to the number of successful text conversions.
V_numConversionsFailed	Set to the number of unsuccessful text conversions.
V_numConversionsSkipped	Set to the number of skipped text conversions. Text conversion may be skipped if the string variable contains binary data and the /SKIP=0 flag is not used. Text conversion is skipped by default if the string variable contains binary data. Text conversion is also skipped by default if newTextEncoding is 1 (UTF-8) and the string variable is already valid as UTF-8. See the /SKIP flag for details.

Examples

// In the following examples 1 means UTF-8, 4 means Shift JIS.

// Convert specific strings' content from Shift JIS to UTF-8
ConvertGlobalStringTextEncoding 4, 1, string0, string1

// Convert all strings' content from Shift JIS to UTF-8
ConvertGlobalStringTextEncoding /DF={root:,1} 4, 1

// Same as before but exclude the root:Packages data folder
ConvertGlobalStringTextEncoding /DF={root:,1,root:Packages} 4, 1

// Convert all strings' content from Shift JIS to UTF-8 except strings containing binary data
ConvertGlobalStringTextEncoding /DF={root:,1}/SKIP=0 4, 1

See Also

Text Encodings, String Variable Text Encodings, Text Encoding Names and Codes

ConvertTextEncoding, SetWaveTextEncoding