[NTG-context] Problem with encoding, XML and writetolist...

[Vit Zyka]

Hans Hagen wrote:
> Vit Zyka wrote:
> 
>>Martin Kolařík wrote:
>>  
>>
>>>Hi Hans,
>>>
>>>I've analyzed the situation using different combinations of regimes and encodings, including switching off handling (\donothandleactivecharacter) and it seems to me, that every text with chars > 127, which is fully expanded (as in 8bit.tex), must cause difficulties, because TOC entry read back from TUO is scanned independently and thus spaces are gobbled.
>>>    
>>
>>Hi Martin,
>>
>>we have discussed this problem a year ago. In that time I solve it just 
>>writing '{}' in each title where was needed, like
>>
>>   \chapter{Dobré{} ráno}
>>
>>Now I have cooked an automatic solution. It is not optimal (effective) 
>>but working. The only needs is to redefine
>>
>>   \def\numcharacter#1{\char#1{}}
>>   \let\dochar\numcharacter
>>
>>and to load appropriate encoding file once more. Since it is banned a 
>>new macro \reuseencoding was introduce:
>>
>>For Hans:
>>It would be nice if this feature was enabled in a standard distro. 
>>Perhaps you will find some more optimal solution...
>>
>>-------------------------------------------------------
>>\unprotected
>>\def\doreuseencoding#1%
>>   {\letvalue{\c!file\f!encodingprefix#1}\empty
>>    \makeshortfilename[\f!encodingprefix#1]%
>>    \startreadingfile
>>    \readsysfile\shortfilename
>>      {\showmessage\m!encodings2{#1}}
>>      {\showmessage\m!encodings3{#1}}%
>>    \stopreadingfile}
>>\def\reuseencoding[#1]%
>>   {\processcommalist[#1]\doreuseencoding}
>>
>>
>>\def\numcharacter#1{\char#1{}}
>>\let\dochar\numcharacter
>>
>>\reuseencoding[ec]
>>
>>\enableregime[latin2]
>>
>>\starttext
>>\writetolist[chapter]{1}{Dobré ráno}
>>\expanded{\writetolist[chapter]{2}{Dobré odpoledne}}
>>
>>\placecontent
>>\stoptext
>>  
> 
> the {} may break liguature building (and maybe kerning) ... taco knows ...

Hmmm, that's right.

> I got a brainwave ... 
> 
> \enableregime[latin2]
> 
> \starttext
> 
> \def\numcharacter#1{\rawcharacter{#1}}%
> \let\dochar\numcharacter

So there is no need to have only 7-bit chars in tui/tuo etc.? All right, 
I think this will be better also for sorting. Sounds promissing.

> \writetolist[chapter]{1}{Dobré ráno}
> 
> \expanded{\writetolist[chapter]{2}{Dobré odpoledne}}
> 
> \placelist[chapter][criterium=text]
> 
> test
> 
> \typefile{test.tuo}
> 
> \stoptext
> 
> this can be sped up (a bit) with 
> 
> % \dostepwiserecurse{0}{255}{1}
> %   {\setevalue{rchr:\recurselevel}{\rawcharacter\recurselevel}}
> 
> % \def\rawcharacter#1{\getvalue{rchr:#1}}
> 
> this means that all chars are round tripped to themselves in output files! 
> 
> raw characters are not active and they are therefore equivalent to \char  
> 
> it's enough then to put 
> 
> \def\numcharacter#1{\rawcharacter{#1}}%
> \let\dochar\numcharacter
> 
> into cont-new.tex 
> 
> (i will discuss this in more detail with taco later) 

All right, thanks Hans,
Vit