Just Solve the File Format Problem - User contributions [en]

Tandy 200 BASIC tokenized file

2024-07-18T04:19:57Z

Hackerb9: /* Calculating PL PH */

{{FormatInfo
|subcat=Source code
|subcat2=Tokenized BASIC
|released=1983
|extensions={{ext|ba}}
}}

'''Tandy 200 BASIC''' was a version of Microsoft BASIC for the Radio Shack Tandy 200 computer. The tokenizations for the TRS-80 Model 100 and Tandy 102 appear to be identical. In fact, while the research that created this page was done on a Tandy 200, experiments using [https://sourceforge.net/projects/virtualt/ emulated systems] indicate that the tokenized BASIC file format is the same across most of the computers related to the Kyocera Kyotronic-85, with the exception being the models made by NEC.

{| class="wikitable" style="margin-left: auto; margin-right: auto; border: none;"
|+ Which of the Kyotronic-85 Sisters use this format
! title="Name of the manufacturer and computer model" | Computer Model
! title="Whether the tokenization is the same as for the Tandy 200" | Uses Tandy 200 Tokenization?
|- style=text-align:center
|Kyocera Kyotronic-85 Olivetti M10 TRS-80 Model 100 Tandy 200 Tandy 102||Yes
|- style=text-align:center
| NEC PC-8201 NEC PC-8201A NEC PC-8300 || No
|}

While the NEC PC-8201/8300 format is not the same, the BASIC it runs (N82 BASIC) is very close and it is expected the tokenization will be similar.

== Tokens ==
All BASIC tokens on the Tandy 200 are a single byte with a value from 128 to 255.

{| class="wikitable"
|+ Table mapping token values to BASIC keywords
! title="The value of the token in hexadecimal" | Hex
! title="The value of the token in decimal" | Dec
! title="The BASIC keyword the token stands for" | Keyword
! title="Miscellany" | Notes
|-
|-
|80||128||END
|-
|81||129||FOR
|-
|82||130||NEXT
|-
|83||131||DATA
|-
|84||132||INPUT
|-
|85||133||DIM
|-
|86||134||READ
|-
|87||135||LET
|-
|88||136||GOTO
|-
|89||137||RUN
|-
|8A||138||IF
|-
|8B||139||RESTORE
|-
|8C||140||GOSUB
|-
|8D||141||RETURN
|-
|8E||142||REM
|-
|8F||143||STOP
|-
|90||144||WIDTH
|-
|91||145||ELSE||When tokenizing, the Tandy 200 always adds a colon (':') before the ELSE token (3A 91). If the user actually writes :ELSE it is tokenized as 3A 3A 91.
|-
|92||146||LINE
|-
|93||147||EDIT
|-
|94||148||ERROR
|-
|95||149||RESUME
|-
|96||150||OUT
|-
|97||151||ON
|-
|98||152||DSKO$
|-
|99||153||OPEN
|-
|9A||154||CLOSE
|-
|9B||155||LOAD||Note that LOADM is simply the token for LOAD followed by an ASCII 'M'.
|-
|9C||156||MERGE
|-QUOTE
|9D||157||FILES
|-
|9E||158||SAVE
|-
|9F||159||LFILES
|-
|A0||160||LPRINT
|-
|A1||161||DEF
|-
|A2||162||POKE
|-
|A3||163||PRINT
|-
|A4||164||CONT
|-
|A5||165||LIST
|-
|A6||166||LLIST
|-
|A7||167||CLEAR
|-
|A8||168||CLOAD
|-
|A9||169||CSAVE
|-
|AA||170||TIME$
|-
|AB||171||DATE$
|-
|AC||172||DAY$
|-
|AD||173||COM
|-
|AE||174||MDM
|-
|AF||175||KEY
|-
|B0||176||CLS
|-
|B1||177||BEEP
|-
|B2||178||SOUND
|-
|B3||179||LCOPY
|-
|B4||180||PSET
|-
|B5||181||PRESET
|-
|B6||182||MOTOR
|-
|B7||183||MAX
|-
|B8||184||POWER
|-
|B9||185||CALL
|-
|BA||186||MENU
|-
|BB||187||IPL
|-
|BC||188||NAME
|-
|BD||189||KILL
|-
|BE||190||SCREEN
|-
|BF||191||NEW
|-
|C0||192||TAB(
|-
|C1||193||TO
|-
|C2||194||USING
|-
|C3||195||VARPTR
|-
|C4||196||ERL
|-
|C5||197||ERR
|-
|C6||198||STRING$
|-
|C7||199||INSTR
|-
|C8||200||DSKI$
|-
|C9||201||INKEY$
|-
|CA||202||CSRLIN
|-
|CB||203||OFF
|-
|CC||204||HIMEM
|-
|CD||205||THEN
|-
|CE||206||NOT
|-
|CF||207||STEP
|-
|D0||208||+
|-
|D1||209||-
|-
|D2||210||*
|-
|D3||211||/
|-
|D4||212||^
|-
|D5||213||AND
|-
|D6||214||OR
|-
|D7||215||XOR
|-
|D8||216||EQV
|-
|D9||217||IMP
|-
|DA||218||MOD
|-
|DB||219||\
|-
|DC||220||>
|-
|DD||221||=
|-
|DE||222||<
|-
|DF||223||SGN
|-
|E0||224||INT
|-
|E1||225||ABS
|-
|E2||226||FRE
|-
|E3||227||INP
|-
|E4||228||LPOS
|-
|E5||229||POS
|-
|E6||230||SQR
|-
|E7||231||RND
|-
|E8||232||LOG
|-
|E9||233||EXP
|-
|EA||234||COS
|-
|EB||235||SIN
|-
|EC||236||TAN
|-
|ED||237||ATN
|-
|EE||238||PEEK
|-
|EF||239||EOF
|-
|F0||240||LOC
|-
|F1||241||LOF
|-
|F2||242||CINT
|-
|F3||243||CSNG
|-
|F4||244||CDBL
|-
|F5||245||FIX
|-
|F6||246||LEN
|-
|F7||247||STR$
|-
|F8||248||VAL
|-
|F9||249||ASC
|-
|FA||250||CHR$
|-
|FB||251||SPACE$
|-
|FC||252||LEFT$
|-
|FD||253||RIGHT$
|-
|FE||254||MID$
|-
|FF||255||' (QUOTE) || When tokenizing, the single quote character expands to three characters: a colon (3A), the byte for REM (8E), and then FF.
|}

== File Format ==

Tokenised BASIC code is a sequence of tokenized lines. Each tokenized line has the following format:

{|class="wikitable"
|+ Format of a single tokenized line
!Name!!Length!!Description
|-
|PL PH
|2 bytes
|Address in RAM of the next line of BASIC; unsigned 16-bit integer, little-endian. In a file, PL PH is merely a placeholder and the value is never used. In RAM, PL PH == 0 marks the end of the tokenized BASIC listing. (See [[#Details for PL PH]].)
|-
|LL LH
|2 bytes
|Line number; unsigned 16-bit integer, little-endian.
|-
|B0 … Bn
|Any number of bytes
|Depending on the high bit, each byte is either a token or an ASCII character.
|-
|NULL
|1 byte
|NULL byte (<code>0x00</code>) to signal end of line.
|}

Typically, the list of tokenized lines continues until the end of the file. Once a program is loaded into RAM, the end is signaled by two consecutive NULLs in the location where PL PH would normally start the next line.

=== Limits ===

* Line numbers can range from 0 to 65529 (but see variances below).
* Tokenized line length is limited to 256 characters, including the trailing NULL byte. (To do: double check this.)

=== Acceptable variances ===

While Tandy BASIC will never generate such files, it has no problem loading a tokenized program that:

* Has line numbers out of order; the numerical value of a line number is ignored. Program lines are always LISTed and RUN in the order they appear in the tokenized file.
* Contains duplicate line numbers; all lines are kept and run in the order found in the file. (GOTO is complicated as search starts at the next line, not the beginning of the program.)
* Contains a line with zero bytes of data; LIST will show the line number with no text afterward. Does not replace or delete a preexisting line with that number.
* Contains arbitrary bytes for PL PH; the value of PL PH is always recalculated at LOAD time.
* Contains line numbers greater than the nominal maximum of 65529; this can be used to store hidden binary data [https://www.mail-archive.com/m100@lists.bitchin100.com/msg16195.html according to John R. Hogerhuis].

''Sidenote'': while these variances are accepted by actual hardware, emulators may not be able to handle such files. For example, [https://sourceforge.net/projects/virtualt/ Virtual T] (as of version 1.7 in 2022) refuses to load tokenized BASIC programs that have lines out of order, duplicate lines, or bogus values for PL PH.

=== Details for PL PH ===
In a file, the specific values of the 16-bit integer PL PH addresses are artifacts of where the file happened to be in memory before saving. The file's values are ''never'' used as Tandy BASIC always recalculates PL PH upon loading a program into RAM.

Although not technically part of the file format, one may want to calculate PL PH for compatibility with broken or persnickety emulators. Expand the section below to see details.

<div class="toccolours mw-collapsible mw-collapsed">
==== Calculating PL PH ====
<div class="mw-collapsible-content">

Once in RAM, PL PH represents the absolute memory address of the next line. According to John Hogerhuis, PL PH is positioned before the line's data so the BASIC interpreter can quickly skip ahead to reach the target line of GOTO, GOSUB, THEN, ELSE, RESTORE, RESUME, RUN, LIST, LLIST, or EDIT.<ref group="note">Searching begins at the ''next'' line in the program. The claim that one can speed up commonly used subroutines by putting them at low numbered lines is plausible as the search would detect immediately that the next line's number was too high and restart the search from the beginning.</ref>)
Additionally, the end of a tokenized BASIC listing is marked by the final line's PL PH equal to zero (two NULL bytes).

Note that because the addresses are absolute, not relative, their values depend not only on the program code but on an initial offset. That offset is why identical programs may not have identical checksums (but see the bacmp program).

By default on the Tandy 200, the first token of the first line of the BASIC program in RAM is at address 0xA001. For all other Model T computers, including the Model 100, the first token is at 0x8001.

----

<references group="note" />

</div>
</div>

== Format documentation ==

* [https://m100.bitchin100.narkive.com/eP2uSl4J/tokenized-basic-programs-bytes-of-mystery Discussion of byte format on Bitchin 100 Mailing List]
* [http://www.club100.org/library/ups/tokens.do BASIC program to create a table of tokens]
* [https://archive.org/details/HiddenPowersOfTheTrs80Model100/page/n229/mode/2up?view=theater Hidden Powers of the TRS 80 Model 100]. Appendix B lists the BASIC token values for the Model 100. Other than the omission of 255 for a single quote remark, this reference matches what hackerb9 found programmatically for the Tandy 200.
* [http://web.archive.org/web/20220324135218/https://help.ayra.ch/trs80-reference#trs-80-model-100-basic-reference Ayra Model 100 BASIC Reference]

== Other links ==

* [https://github.com/hackerb9/tokenize/ Hackerb9's C program for tokenizing BASIC code]
* [https://sourceforge.net/projects/virtualt/ A Tandy 200 & Model 100/102 emulator]
* [http://bitchin100.com/CloudT/ A TRS-80 Model 100 emulator that runs in a web browser]

[[Category:TRS-80]]
[[Category:Microsoft]]

Tandy 200 BASIC tokenized file

2024-07-18T04:18:02Z

Hackerb9: /* Details for PL PH */ Fill in more details, make section collapsed by default as it will grow longer once an example is added

{{FormatInfo
|subcat=Source code
|subcat2=Tokenized BASIC
|released=1983
|extensions={{ext|ba}}
}}

'''Tandy 200 BASIC''' was a version of Microsoft BASIC for the Radio Shack Tandy 200 computer. The tokenizations for the TRS-80 Model 100 and Tandy 102 appear to be identical. In fact, while the research that created this page was done on a Tandy 200, experiments using [https://sourceforge.net/projects/virtualt/ emulated systems] indicate that the tokenized BASIC file format is the same across most of the computers related to the Kyocera Kyotronic-85, with the exception being the models made by NEC.

{| class="wikitable" style="margin-left: auto; margin-right: auto; border: none;"
|+ Which of the Kyotronic-85 Sisters use this format
! title="Name of the manufacturer and computer model" | Computer Model
! title="Whether the tokenization is the same as for the Tandy 200" | Uses Tandy 200 Tokenization?
|- style=text-align:center
|Kyocera Kyotronic-85 Olivetti M10 TRS-80 Model 100 Tandy 200 Tandy 102||Yes
|- style=text-align:center
| NEC PC-8201 NEC PC-8201A NEC PC-8300 || No
|}

While the NEC PC-8201/8300 format is not the same, the BASIC it runs (N82 BASIC) is very close and it is expected the tokenization will be similar.

== Tokens ==
All BASIC tokens on the Tandy 200 are a single byte with a value from 128 to 255.

{| class="wikitable"
|+ Table mapping token values to BASIC keywords
! title="The value of the token in hexadecimal" | Hex
! title="The value of the token in decimal" | Dec
! title="The BASIC keyword the token stands for" | Keyword
! title="Miscellany" | Notes
|-
|-
|80||128||END
|-
|81||129||FOR
|-
|82||130||NEXT
|-
|83||131||DATA
|-
|84||132||INPUT
|-
|85||133||DIM
|-
|86||134||READ
|-
|87||135||LET
|-
|88||136||GOTO
|-
|89||137||RUN
|-
|8A||138||IF
|-
|8B||139||RESTORE
|-
|8C||140||GOSUB
|-
|8D||141||RETURN
|-
|8E||142||REM
|-
|8F||143||STOP
|-
|90||144||WIDTH
|-
|91||145||ELSE||When tokenizing, the Tandy 200 always adds a colon (':') before the ELSE token (3A 91). If the user actually writes :ELSE it is tokenized as 3A 3A 91.
|-
|92||146||LINE
|-
|93||147||EDIT
|-
|94||148||ERROR
|-
|95||149||RESUME
|-
|96||150||OUT
|-
|97||151||ON
|-
|98||152||DSKO$
|-
|99||153||OPEN
|-
|9A||154||CLOSE
|-
|9B||155||LOAD||Note that LOADM is simply the token for LOAD followed by an ASCII 'M'.
|-
|9C||156||MERGE
|-QUOTE
|9D||157||FILES
|-
|9E||158||SAVE
|-
|9F||159||LFILES
|-
|A0||160||LPRINT
|-
|A1||161||DEF
|-
|A2||162||POKE
|-
|A3||163||PRINT
|-
|A4||164||CONT
|-
|A5||165||LIST
|-
|A6||166||LLIST
|-
|A7||167||CLEAR
|-
|A8||168||CLOAD
|-
|A9||169||CSAVE
|-
|AA||170||TIME$
|-
|AB||171||DATE$
|-
|AC||172||DAY$
|-
|AD||173||COM
|-
|AE||174||MDM
|-
|AF||175||KEY
|-
|B0||176||CLS
|-
|B1||177||BEEP
|-
|B2||178||SOUND
|-
|B3||179||LCOPY
|-
|B4||180||PSET
|-
|B5||181||PRESET
|-
|B6||182||MOTOR
|-
|B7||183||MAX
|-
|B8||184||POWER
|-
|B9||185||CALL
|-
|BA||186||MENU
|-
|BB||187||IPL
|-
|BC||188||NAME
|-
|BD||189||KILL
|-
|BE||190||SCREEN
|-
|BF||191||NEW
|-
|C0||192||TAB(
|-
|C1||193||TO
|-
|C2||194||USING
|-
|C3||195||VARPTR
|-
|C4||196||ERL
|-
|C5||197||ERR
|-
|C6||198||STRING$
|-
|C7||199||INSTR
|-
|C8||200||DSKI$
|-
|C9||201||INKEY$
|-
|CA||202||CSRLIN
|-
|CB||203||OFF
|-
|CC||204||HIMEM
|-
|CD||205||THEN
|-
|CE||206||NOT
|-
|CF||207||STEP
|-
|D0||208||+
|-
|D1||209||-
|-
|D2||210||*
|-
|D3||211||/
|-
|D4||212||^
|-
|D5||213||AND
|-
|D6||214||OR
|-
|D7||215||XOR
|-
|D8||216||EQV
|-
|D9||217||IMP
|-
|DA||218||MOD
|-
|DB||219||\
|-
|DC||220||>
|-
|DD||221||=
|-
|DE||222||<
|-
|DF||223||SGN
|-
|E0||224||INT
|-
|E1||225||ABS
|-
|E2||226||FRE
|-
|E3||227||INP
|-
|E4||228||LPOS
|-
|E5||229||POS
|-
|E6||230||SQR
|-
|E7||231||RND
|-
|E8||232||LOG
|-
|E9||233||EXP
|-
|EA||234||COS
|-
|EB||235||SIN
|-
|EC||236||TAN
|-
|ED||237||ATN
|-
|EE||238||PEEK
|-
|EF||239||EOF
|-
|F0||240||LOC
|-
|F1||241||LOF
|-
|F2||242||CINT
|-
|F3||243||CSNG
|-
|F4||244||CDBL
|-
|F5||245||FIX
|-
|F6||246||LEN
|-
|F7||247||STR$
|-
|F8||248||VAL
|-
|F9||249||ASC
|-
|FA||250||CHR$
|-
|FB||251||SPACE$
|-
|FC||252||LEFT$
|-
|FD||253||RIGHT$
|-
|FE||254||MID$
|-
|FF||255||' (QUOTE) || When tokenizing, the single quote character expands to three characters: a colon (3A), the byte for REM (8E), and then FF.
|}

== File Format ==

Tokenised BASIC code is a sequence of tokenized lines. Each tokenized line has the following format:

{|class="wikitable"
|+ Format of a single tokenized line
!Name!!Length!!Description
|-
|PL PH
|2 bytes
|Address in RAM of the next line of BASIC; unsigned 16-bit integer, little-endian. In a file, PL PH is merely a placeholder and the value is never used. In RAM, PL PH == 0 marks the end of the tokenized BASIC listing. (See [[#Details for PL PH]].)
|-
|LL LH
|2 bytes
|Line number; unsigned 16-bit integer, little-endian.
|-
|B0 … Bn
|Any number of bytes
|Depending on the high bit, each byte is either a token or an ASCII character.
|-
|NULL
|1 byte
|NULL byte (<code>0x00</code>) to signal end of line.
|}

Typically, the list of tokenized lines continues until the end of the file. Once a program is loaded into RAM, the end is signaled by two consecutive NULLs in the location where PL PH would normally start the next line.

=== Limits ===

* Line numbers can range from 0 to 65529 (but see variances below).
* Tokenized line length is limited to 256 characters, including the trailing NULL byte. (To do: double check this.)

=== Acceptable variances ===

While Tandy BASIC will never generate such files, it has no problem loading a tokenized program that:

* Has line numbers out of order; the numerical value of a line number is ignored. Program lines are always LISTed and RUN in the order they appear in the tokenized file.
* Contains duplicate line numbers; all lines are kept and run in the order found in the file. (GOTO is complicated as search starts at the next line, not the beginning of the program.)
* Contains a line with zero bytes of data; LIST will show the line number with no text afterward. Does not replace or delete a preexisting line with that number.
* Contains arbitrary bytes for PL PH; the value of PL PH is always recalculated at LOAD time.
* Contains line numbers greater than the nominal maximum of 65529; this can be used to store hidden binary data [https://www.mail-archive.com/m100@lists.bitchin100.com/msg16195.html according to John R. Hogerhuis].

''Sidenote'': while these variances are accepted by actual hardware, emulators may not be able to handle such files. For example, [https://sourceforge.net/projects/virtualt/ Virtual T] (as of version 1.7 in 2022) refuses to load tokenized BASIC programs that have lines out of order, duplicate lines, or bogus values for PL PH.

=== Details for PL PH ===
In a file, the specific values of the 16-bit integer PL PH addresses are artifacts of where the file happened to be in memory before saving. The file's values are ''never'' used as Tandy BASIC always recalculates PL PH upon loading a program into RAM.

Although not technically part of the file format, one may want to calculate PL PH for compatibility with broken or persnickety emulators. Expand the section below to see details.

<div class="toccolours mw-collapsible mw-collapsed">
==== Calculating PL PH ====
<div class="mw-collapsible-content">

Once in RAM, PL PH represents the absolute memory address of the next line. According to John Hogerhuis, PL PH is positioned before the line's data so the BASIC interpreter can quickly skip ahead to reach the target line of GOTO, GOSUB, THEN, ELSE, RESTORE, RESUME, RUN, LIST, LLIST, or EDIT.<ref group="note">Searching begins at the ''next'' line in the program. The claim that one can speed up commonly used subroutines by putting them at low numbered lines is plausible as the search would detect immediately that the next line's number was too high and restart the search from the beginning.</ref>)
Additionally, the end of a tokenized BASIC listing is marked by the final line's PL PH equal to zero (two NULL bytes).

Note that because the addresses are absolute, not relative, their values depend not only on the program code but on an initial offset. That offset is why identical programs may not have identical checksums (but see the bacmp program).

By default on the Tandy 200, the first token of the first line of the BASIC program in RAM is at address 0xA001. For all other Model T computers, including the Model 100, the first token is at 0x8001.

<references group="note" />

</div>
</div>

== Format documentation ==

* [https://m100.bitchin100.narkive.com/eP2uSl4J/tokenized-basic-programs-bytes-of-mystery Discussion of byte format on Bitchin 100 Mailing List]
* [http://www.club100.org/library/ups/tokens.do BASIC program to create a table of tokens]
* [https://archive.org/details/HiddenPowersOfTheTrs80Model100/page/n229/mode/2up?view=theater Hidden Powers of the TRS 80 Model 100]. Appendix B lists the BASIC token values for the Model 100. Other than the omission of 255 for a single quote remark, this reference matches what hackerb9 found programmatically for the Tandy 200.
* [http://web.archive.org/web/20220324135218/https://help.ayra.ch/trs80-reference#trs-80-model-100-basic-reference Ayra Model 100 BASIC Reference]

== Other links ==

* [https://github.com/hackerb9/tokenize/ Hackerb9's C program for tokenizing BASIC code]
* [https://sourceforge.net/projects/virtualt/ A Tandy 200 & Model 100/102 emulator]
* [http://bitchin100.com/CloudT/ A TRS-80 Model 100 emulator that runs in a web browser]

[[Category:TRS-80]]
[[Category:Microsoft]]

Tandy 200 BASIC tokenized file

2024-03-18T06:37:41Z

Hackerb9:

{{FormatInfo
|subcat=Source code
|subcat2=Tokenized BASIC
|released=1983
|extensions={{ext|ba}}
}}

'''Tandy 200 BASIC''' was a version of Microsoft BASIC for the Radio Shack Tandy 200 computer. The tokenizations for the TRS-80 Model 100 and Tandy 102 appear to be identical. In fact, while the research that created this page was done on a Tandy 200, experiments using [https://sourceforge.net/projects/virtualt/ emulated systems] indicate that the tokenized BASIC file format is the same across most of the computers related to the Kyocera Kyotronic-85, with the exception being the models made by NEC.

{| class="wikitable" style="margin-left: auto; margin-right: auto; border: none;"
|+ Which of the Kyotronic-85 Sisters use this format
! title="Name of the manufacturer and computer model" | Computer Model
! title="Whether the tokenization is the same as for the Tandy 200" | Uses Tandy 200 Tokenization?
|- style=text-align:center
|Kyocera Kyotronic-85 Olivetti M10 TRS-80 Model 100 Tandy 200 Tandy 102||Yes
|- style=text-align:center
| NEC PC-8201 NEC PC-8201A NEC PC-8300 || No
|}

While the NEC PC-8201/8300 format is not the same, the BASIC it runs (N82 BASIC) is very close and it is expected the tokenization will be similar.

== Tokens ==
All BASIC tokens on the Tandy 200 are a single byte with a value from 128 to 255.

{| class="wikitable"
|+ Table mapping token values to BASIC keywords
! title="The value of the token in hexadecimal" | Hex
! title="The value of the token in decimal" | Dec
! title="The BASIC keyword the token stands for" | Keyword
! title="Miscellany" | Notes
|-
|-
|80||128||END
|-
|81||129||FOR
|-
|82||130||NEXT
|-
|83||131||DATA
|-
|84||132||INPUT
|-
|85||133||DIM
|-
|86||134||READ
|-
|87||135||LET
|-
|88||136||GOTO
|-
|89||137||RUN
|-
|8A||138||IF
|-
|8B||139||RESTORE
|-
|8C||140||GOSUB
|-
|8D||141||RETURN
|-
|8E||142||REM
|-
|8F||143||STOP
|-
|90||144||WIDTH
|-
|91||145||ELSE||When tokenizing, the Tandy 200 always adds a colon (':') before the ELSE token (3A 91). If the user actually writes :ELSE it is tokenized as 3A 3A 91.
|-
|92||146||LINE
|-
|93||147||EDIT
|-
|94||148||ERROR
|-
|95||149||RESUME
|-
|96||150||OUT
|-
|97||151||ON
|-
|98||152||DSKO$
|-
|99||153||OPEN
|-
|9A||154||CLOSE
|-
|9B||155||LOAD||Note that LOADM is simply the token for LOAD followed by an ASCII 'M'.
|-
|9C||156||MERGE
|-QUOTE
|9D||157||FILES
|-
|9E||158||SAVE
|-
|9F||159||LFILES
|-
|A0||160||LPRINT
|-
|A1||161||DEF
|-
|A2||162||POKE
|-
|A3||163||PRINT
|-
|A4||164||CONT
|-
|A5||165||LIST
|-
|A6||166||LLIST
|-
|A7||167||CLEAR
|-
|A8||168||CLOAD
|-
|A9||169||CSAVE
|-
|AA||170||TIME$
|-
|AB||171||DATE$
|-
|AC||172||DAY$
|-
|AD||173||COM
|-
|AE||174||MDM
|-
|AF||175||KEY
|-
|B0||176||CLS
|-
|B1||177||BEEP
|-
|B2||178||SOUND
|-
|B3||179||LCOPY
|-
|B4||180||PSET
|-
|B5||181||PRESET
|-
|B6||182||MOTOR
|-
|B7||183||MAX
|-
|B8||184||POWER
|-
|B9||185||CALL
|-
|BA||186||MENU
|-
|BB||187||IPL
|-
|BC||188||NAME
|-
|BD||189||KILL
|-
|BE||190||SCREEN
|-
|BF||191||NEW
|-
|C0||192||TAB(
|-
|C1||193||TO
|-
|C2||194||USING
|-
|C3||195||VARPTR
|-
|C4||196||ERL
|-
|C5||197||ERR
|-
|C6||198||STRING$
|-
|C7||199||INSTR
|-
|C8||200||DSKI$
|-
|C9||201||INKEY$
|-
|CA||202||CSRLIN
|-
|CB||203||OFF
|-
|CC||204||HIMEM
|-
|CD||205||THEN
|-
|CE||206||NOT
|-
|CF||207||STEP
|-
|D0||208||+
|-
|D1||209||-
|-
|D2||210||*
|-
|D3||211||/
|-
|D4||212||^
|-
|D5||213||AND
|-
|D6||214||OR
|-
|D7||215||XOR
|-
|D8||216||EQV
|-
|D9||217||IMP
|-
|DA||218||MOD
|-
|DB||219||\
|-
|DC||220||>
|-
|DD||221||=
|-
|DE||222||<
|-
|DF||223||SGN
|-
|E0||224||INT
|-
|E1||225||ABS
|-
|E2||226||FRE
|-
|E3||227||INP
|-
|E4||228||LPOS
|-
|E5||229||POS
|-
|E6||230||SQR
|-
|E7||231||RND
|-
|E8||232||LOG
|-
|E9||233||EXP
|-
|EA||234||COS
|-
|EB||235||SIN
|-
|EC||236||TAN
|-
|ED||237||ATN
|-
|EE||238||PEEK
|-
|EF||239||EOF
|-
|F0||240||LOC
|-
|F1||241||LOF
|-
|F2||242||CINT
|-
|F3||243||CSNG
|-
|F4||244||CDBL
|-
|F5||245||FIX
|-
|F6||246||LEN
|-
|F7||247||STR$
|-
|F8||248||VAL
|-
|F9||249||ASC
|-
|FA||250||CHR$
|-
|FB||251||SPACE$
|-
|FC||252||LEFT$
|-
|FD||253||RIGHT$
|-
|FE||254||MID$
|-
|FF||255||' (QUOTE) || When tokenizing, the single quote character expands to three characters: a colon (3A), the byte for REM (8E), and then FF.
|}

== File Format ==

Tokenised BASIC code is a sequence of tokenized lines. Each tokenized line has the following format:

{|class="wikitable"
|+ Format of a single tokenized line
!Name!!Length!!Description
|-
|PL PH
|2 bytes
|Address in RAM of the next line of BASIC; unsigned 16-bit integer, little-endian. In a file, PL PH is merely a placeholder and the value is never used. In RAM, PL PH == 0 marks the end of the tokenized BASIC listing. (See [[#Details for PL PH]].)
|-
|LL LH
|2 bytes
|Line number; unsigned 16-bit integer, little-endian.
|-
|B0 … Bn
|Any number of bytes
|Depending on the high bit, each byte is either a token or an ASCII character.
|-
|NULL
|1 byte
|NULL byte (<code>0x00</code>) to signal end of line.
|}

Typically, the list of tokenized lines continues until the end of the file. Once a program is loaded into RAM, the end is signaled by two consecutive NULLs in the location where PL PH would normally start the next line.

=== Limits ===

* Line numbers can range from 0 to 65529 (but see variances below).
* Tokenized line length is limited to 256 characters, including the trailing NULL byte. (To do: double check this.)

=== Acceptable variances ===

While Tandy BASIC will never generate such files, it has no problem loading a tokenized program that:

* Has line numbers out of order; the numerical value of a line number is ignored. Program lines are always LISTed and RUN in the order they appear in the tokenized file.
* Contains duplicate line numbers; all lines are kept and run in the order found in the file. (GOTO is complicated as search starts at the next line, not the beginning of the program.)
* Contains a line with zero bytes of data; LIST will show the line number with no text afterward. Does not replace or delete a preexisting line with that number.
* Contains arbitrary bytes for PL PH; the value of PL PH is always recalculated at LOAD time.
* Contains line numbers greater than the nominal maximum of 65529; this can be used to store hidden binary data [https://www.mail-archive.com/m100@lists.bitchin100.com/msg16195.html according to John R. Hogerhuis].

''Sidenote'': while these variances are accepted by actual hardware, emulators may not be able to handle such files. For example, [https://sourceforge.net/projects/virtualt/ Virtual T] (as of version 1.7 in 2022) refuses to load tokenized BASIC programs that have lines out of order, duplicate lines, or bogus values for PL PH.

=== Details for PL PH ===
In a file, the specific values of the 16-bit integer PL PH are ''never'' used. Tandy BASIC always recalculates PL PH before loading a program into RAM. Although not technically part of the file format, one may want to calculate PL PH for compatibility with broken or persnickety emulators.

<div class="toccolours mw-collapsible mw-collapsed">
==== Calculating PL PH ====
<div class="mw-collapsible-content">
Once in RAM, PL PH represents the memory address of the next line. According to John Hogerhuis, the purpose is to quickly skip through lines in order to reach the target of GOTO, GOSUB, and READ. Additionally, the end of a tokenized BASIC listing is marked by the final line's PL PH equal to zero (two NULL bytes).

By default on the Tandy 200, the first token of the first line of the BASIC program in RAM is at address 0xA001.
</div>
</div>

== Format documentation ==

* [https://m100.bitchin100.narkive.com/eP2uSl4J/tokenized-basic-programs-bytes-of-mystery Discussion of byte format on Bitchin 100 Mailing List]
* [http://www.club100.org/library/ups/tokens.do BASIC program to create a table of tokens]
* [https://archive.org/details/HiddenPowersOfTheTrs80Model100/page/n229/mode/2up?view=theater Hidden Powers of the TRS 80 Model 100]. Appendix B lists the BASIC token values for the Model 100. Other than the omission of 255 for a single quote remark, this reference matches what hackerb9 found programmatically for the Tandy 200.
* [http://web.archive.org/web/20220324135218/https://help.ayra.ch/trs80-reference#trs-80-model-100-basic-reference Ayra Model 100 BASIC Reference]

== Other links ==

* [https://github.com/hackerb9/tokenize/ Hackerb9's C program for tokenizing BASIC code]
* [https://sourceforge.net/projects/virtualt/ A Tandy 200 & Model 100/102 emulator]
* [http://bitchin100.com/CloudT/ A TRS-80 Model 100 emulator that runs in a web browser]

[[Category:TRS-80]]
[[Category:Microsoft]]

Tandy 200 BASIC tokenized file

2024-03-18T06:23:02Z

Hackerb9: Add .BA category

{{FormatInfo
|subcat=Source code
|subcat2=Tokenized BASIC
|released=1983
}}

'''Tandy 200 BASIC''' was a version of Microsoft BASIC for the Radio Shack Tandy 200 computer. The tokenizations for the TRS-80 Model 100 and Tandy 102 appear to be identical. In fact, while the research that created this page was done on a Tandy 200, experiments using [https://sourceforge.net/projects/virtualt/ emulated systems] indicate that the tokenized BASIC file format is the same across most of the computers related to the Kyocera Kyotronic-85, with the exception being the models made by NEC.

{| class="wikitable" style="margin-left: auto; margin-right: auto; border: none;"
|+ Which of the Kyotronic-85 Sisters use this format
! title="Name of the manufacturer and computer model" | Computer Model
! title="Whether the tokenization is the same as for the Tandy 200" | Uses Tandy 200 Tokenization?
|- style=text-align:center
|Kyocera Kyotronic-85 Olivetti M10 TRS-80 Model 100 Tandy 200 Tandy 102||Yes
|- style=text-align:center
| NEC PC-8201 NEC PC-8201A NEC PC-8300 || No
|extensions={{ext|ba}}
|}

While the NEC PC-8201/8300 format is not the same, the BASIC it runs (N82 BASIC) is very close and it is expected the tokenization will be similar.

== Tokens ==
All BASIC tokens on the Tandy 200 are a single byte with a value from 128 to 255.

{| class="wikitable"
|+ Table mapping token values to BASIC keywords
! title="The value of the token in hexadecimal" | Hex
! title="The value of the token in decimal" | Dec
! title="The BASIC keyword the token stands for" | Keyword
! title="Miscellany" | Notes
|-
|-
|80||128||END
|-
|81||129||FOR
|-
|82||130||NEXT
|-
|83||131||DATA
|-
|84||132||INPUT
|-
|85||133||DIM
|-
|86||134||READ
|-
|87||135||LET
|-
|88||136||GOTO
|-
|89||137||RUN
|-
|8A||138||IF
|-
|8B||139||RESTORE
|-
|8C||140||GOSUB
|-
|8D||141||RETURN
|-
|8E||142||REM
|-
|8F||143||STOP
|-
|90||144||WIDTH
|-
|91||145||ELSE||When tokenizing, the Tandy 200 always adds a colon (':') before the ELSE token (3A 91). If the user actually writes :ELSE it is tokenized as 3A 3A 91.
|-
|92||146||LINE
|-
|93||147||EDIT
|-
|94||148||ERROR
|-
|95||149||RESUME
|-
|96||150||OUT
|-
|97||151||ON
|-
|98||152||DSKO$
|-
|99||153||OPEN
|-
|9A||154||CLOSE
|-
|9B||155||LOAD||Note that LOADM is simply the token for LOAD followed by an ASCII 'M'.
|-
|9C||156||MERGE
|-QUOTE
|9D||157||FILES
|-
|9E||158||SAVE
|-
|9F||159||LFILES
|-
|A0||160||LPRINT
|-
|A1||161||DEF
|-
|A2||162||POKE
|-
|A3||163||PRINT
|-
|A4||164||CONT
|-
|A5||165||LIST
|-
|A6||166||LLIST
|-
|A7||167||CLEAR
|-
|A8||168||CLOAD
|-
|A9||169||CSAVE
|-
|AA||170||TIME$
|-
|AB||171||DATE$
|-
|AC||172||DAY$
|-
|AD||173||COM
|-
|AE||174||MDM
|-
|AF||175||KEY
|-
|B0||176||CLS
|-
|B1||177||BEEP
|-
|B2||178||SOUND
|-
|B3||179||LCOPY
|-
|B4||180||PSET
|-
|B5||181||PRESET
|-
|B6||182||MOTOR
|-
|B7||183||MAX
|-
|B8||184||POWER
|-
|B9||185||CALL
|-
|BA||186||MENU
|-
|BB||187||IPL
|-
|BC||188||NAME
|-
|BD||189||KILL
|-
|BE||190||SCREEN
|-
|BF||191||NEW
|-
|C0||192||TAB(
|-
|C1||193||TO
|-
|C2||194||USING
|-
|C3||195||VARPTR
|-
|C4||196||ERL
|-
|C5||197||ERR
|-
|C6||198||STRING$
|-
|C7||199||INSTR
|-
|C8||200||DSKI$
|-
|C9||201||INKEY$
|-
|CA||202||CSRLIN
|-
|CB||203||OFF
|-
|CC||204||HIMEM
|-
|CD||205||THEN
|-
|CE||206||NOT
|-
|CF||207||STEP
|-
|D0||208||+
|-
|D1||209||-
|-
|D2||210||*
|-
|D3||211||/
|-
|D4||212||^
|-
|D5||213||AND
|-
|D6||214||OR
|-
|D7||215||XOR
|-
|D8||216||EQV
|-
|D9||217||IMP
|-
|DA||218||MOD
|-
|DB||219||\
|-
|DC||220||>
|-
|DD||221||=
|-
|DE||222||<
|-
|DF||223||SGN
|-
|E0||224||INT
|-
|E1||225||ABS
|-
|E2||226||FRE
|-
|E3||227||INP
|-
|E4||228||LPOS
|-
|E5||229||POS
|-
|E6||230||SQR
|-
|E7||231||RND
|-
|E8||232||LOG
|-
|E9||233||EXP
|-
|EA||234||COS
|-
|EB||235||SIN
|-
|EC||236||TAN
|-
|ED||237||ATN
|-
|EE||238||PEEK
|-
|EF||239||EOF
|-
|F0||240||LOC
|-
|F1||241||LOF
|-
|F2||242||CINT
|-
|F3||243||CSNG
|-
|F4||244||CDBL
|-
|F5||245||FIX
|-
|F6||246||LEN
|-
|F7||247||STR$
|-
|F8||248||VAL
|-
|F9||249||ASC
|-
|FA||250||CHR$
|-
|FB||251||SPACE$
|-
|FC||252||LEFT$
|-
|FD||253||RIGHT$
|-
|FE||254||MID$
|-
|FF||255||' (QUOTE) || When tokenizing, the single quote character expands to three characters: a colon (3A), the byte for REM (8E), and then FF.
|}

== File Format ==

Tokenised BASIC code is a sequence of tokenized lines. Each tokenized line has the following format:

{|class="wikitable"
|+ Format of a single tokenized line
!Name!!Length!!Description
|-
|PL PH
|2 bytes
|Address in RAM of the next line of BASIC; unsigned 16-bit integer, little-endian. In a file, PL PH is merely a placeholder and the value is never used. In RAM, PL PH == 0 marks the end of the tokenized BASIC listing. (See [[#Details for PL PH]].)
|-
|LL LH
|2 bytes
|Line number; unsigned 16-bit integer, little-endian.
|-
|B0 … Bn
|Any number of bytes
|Depending on the high bit, each byte is either a token or an ASCII character.
|-
|NULL
|1 byte
|NULL byte (<code>0x00</code>) to signal end of line.
|}

Typically, the list of tokenized lines continues until the end of the file. Once a program is loaded into RAM, the end is signaled by two consecutive NULLs in the location where PL PH would normally start the next line.

=== Limits ===

* Line numbers can range from 0 to 65529 (but see variances below).
* Tokenized line length is limited to 256 characters, including the trailing NULL byte. (To do: double check this.)

=== Acceptable variances ===

While Tandy BASIC will never generate such files, it has no problem loading a tokenized program that:

* Has line numbers out of order; the numerical value of a line number is ignored. Program lines are always LISTed and RUN in the order they appear in the tokenized file.
* Contains duplicate line numbers; all lines are kept and run in the order found in the file. (GOTO is complicated as search starts at the next line, not the beginning of the program.)
* Contains a line with zero bytes of data; LIST will show the line number with no text afterward. Does not replace or delete a preexisting line with that number.
* Contains arbitrary bytes for PL PH; the value of PL PH is always recalculated at LOAD time.
* Contains line numbers greater than the nominal maximum of 65529; this can be used to store hidden binary data [https://www.mail-archive.com/m100@lists.bitchin100.com/msg16195.html according to John R. Hogerhuis].

''Sidenote'': while these variances are accepted by actual hardware, emulators may not be able to handle such files. For example, [https://sourceforge.net/projects/virtualt/ Virtual T] (as of version 1.7 in 2022) refuses to load tokenized BASIC programs that have lines out of order, duplicate lines, or bogus values for PL PH.

=== Details for PL PH ===
In a file, the specific values of the 16-bit integer PL PH are ''never'' used. Tandy BASIC always recalculates PL PH before loading a program into RAM. Although not technically part of the file format, one may want to calculate PL PH for compatibility with broken or persnickety emulators.

<div class="toccolours mw-collapsible mw-collapsed">
==== Calculating PL PH ====
<div class="mw-collapsible-content">
Once in RAM, PL PH represents the memory address of the next line. According to John Hogerhuis, the purpose is to quickly skip through lines in order to reach the target of GOTO, GOSUB, and READ. Additionally, the end of a tokenized BASIC listing is marked by the final line's PL PH equal to zero (two NULL bytes).

By default on the Tandy 200, the first token of the first line of the BASIC program in RAM is at address 0xA001.
</div>
</div>

== Format documentation ==

* [https://m100.bitchin100.narkive.com/eP2uSl4J/tokenized-basic-programs-bytes-of-mystery Discussion of byte format on Bitchin 100 Mailing List]
* [http://www.club100.org/library/ups/tokens.do BASIC program to create a table of tokens]
* [https://archive.org/details/HiddenPowersOfTheTrs80Model100/page/n229/mode/2up?view=theater Hidden Powers of the TRS 80 Model 100]. Appendix B lists the BASIC token values for the Model 100. Other than the omission of 255 for a single quote remark, this reference matches what hackerb9 found programmatically for the Tandy 200.
* [http://web.archive.org/web/20220324135218/https://help.ayra.ch/trs80-reference#trs-80-model-100-basic-reference Ayra Model 100 BASIC Reference]

== Other links ==

* [https://github.com/hackerb9/tokenize/ Hackerb9's C program for tokenizing BASIC code]
* [https://sourceforge.net/projects/virtualt/ A Tandy 200 & Model 100/102 emulator]
* [http://bitchin100.com/CloudT/ A TRS-80 Model 100 emulator that runs in a web browser]

[[Category:TRS-80]]
[[Category:Microsoft]]

Tandy 200 BASIC tokenized file

2024-03-11T02:28:02Z

Hackerb9: /* Calculating PL PH */ CLEAR / HIMEM is for something else.

{{FormatInfo
|subcat=Source code
|subcat2=Tokenized BASIC
|released=1983
}}

'''Tandy 200 BASIC''' was a version of Microsoft BASIC for the Radio Shack Tandy 200 computer. The tokenizations for the TRS-80 Model 100 and Tandy 102 appear to be identical. In fact, while the research that created this page was done on a Tandy 200, experiments using [https://sourceforge.net/projects/virtualt/ emulated systems] indicate that the tokenized BASIC file format is the same across most of the computers related to the Kyocera Kyotronic-85, with the exception being the models made by NEC.

{| class="wikitable" style="margin-left: auto; margin-right: auto; border: none;"
|+ Which of the Kyotronic-85 Sisters use this format
! title="Name of the manufacturer and computer model" | Computer Model
! title="Whether the tokenization is the same as for the Tandy 200" | Uses Tandy 200 Tokenization?
|- style=text-align:center
|Kyocera Kyotronic-85 Olivetti M10 TRS-80 Model 100 Tandy 200 Tandy 102||Yes
|- style=text-align:center
| NEC PC-8201 NEC PC-8201A NEC PC-8300 || No
|}

While the NEC PC-8201/8300 format is not the same, the BASIC it runs (N82 BASIC) is very close and it is expected the tokenization will be similar.

== Tokens ==
All BASIC tokens on the Tandy 200 are a single byte with a value from 128 to 255.

{| class="wikitable"
|+ Table mapping token values to BASIC keywords
! title="The value of the token in hexadecimal" | Hex
! title="The value of the token in decimal" | Dec
! title="The BASIC keyword the token stands for" | Keyword
! title="Miscellany" | Notes
|-
|-
|80||128||END
|-
|81||129||FOR
|-
|82||130||NEXT
|-
|83||131||DATA
|-
|84||132||INPUT
|-
|85||133||DIM
|-
|86||134||READ
|-
|87||135||LET
|-
|88||136||GOTO
|-
|89||137||RUN
|-
|8A||138||IF
|-
|8B||139||RESTORE
|-
|8C||140||GOSUB
|-
|8D||141||RETURN
|-
|8E||142||REM
|-
|8F||143||STOP
|-
|90||144||WIDTH
|-
|91||145||ELSE||When tokenizing, the Tandy 200 always adds a colon (':') before the ELSE token (3A 91). If the user actually writes :ELSE it is tokenized as 3A 3A 91.
|-
|92||146||LINE
|-
|93||147||EDIT
|-
|94||148||ERROR
|-
|95||149||RESUME
|-
|96||150||OUT
|-
|97||151||ON
|-
|98||152||DSKO$
|-
|99||153||OPEN
|-
|9A||154||CLOSE
|-
|9B||155||LOAD||Note that LOADM is simply the token for LOAD followed by an ASCII 'M'.
|-
|9C||156||MERGE
|-QUOTE
|9D||157||FILES
|-
|9E||158||SAVE
|-
|9F||159||LFILES
|-
|A0||160||LPRINT
|-
|A1||161||DEF
|-
|A2||162||POKE
|-
|A3||163||PRINT
|-
|A4||164||CONT
|-
|A5||165||LIST
|-
|A6||166||LLIST
|-
|A7||167||CLEAR
|-
|A8||168||CLOAD
|-
|A9||169||CSAVE
|-
|AA||170||TIME$
|-
|AB||171||DATE$
|-
|AC||172||DAY$
|-
|AD||173||COM
|-
|AE||174||MDM
|-
|AF||175||KEY
|-
|B0||176||CLS
|-
|B1||177||BEEP
|-
|B2||178||SOUND
|-
|B3||179||LCOPY
|-
|B4||180||PSET
|-
|B5||181||PRESET
|-
|B6||182||MOTOR
|-
|B7||183||MAX
|-
|B8||184||POWER
|-
|B9||185||CALL
|-
|BA||186||MENU
|-
|BB||187||IPL
|-
|BC||188||NAME
|-
|BD||189||KILL
|-
|BE||190||SCREEN
|-
|BF||191||NEW
|-
|C0||192||TAB(
|-
|C1||193||TO
|-
|C2||194||USING
|-
|C3||195||VARPTR
|-
|C4||196||ERL
|-
|C5||197||ERR
|-
|C6||198||STRING$
|-
|C7||199||INSTR
|-
|C8||200||DSKI$
|-
|C9||201||INKEY$
|-
|CA||202||CSRLIN
|-
|CB||203||OFF
|-
|CC||204||HIMEM
|-
|CD||205||THEN
|-
|CE||206||NOT
|-
|CF||207||STEP
|-
|D0||208||+
|-
|D1||209||-
|-
|D2||210||*
|-
|D3||211||/
|-
|D4||212||^
|-
|D5||213||AND
|-
|D6||214||OR
|-
|D7||215||XOR
|-
|D8||216||EQV
|-
|D9||217||IMP
|-
|DA||218||MOD
|-
|DB||219||\
|-
|DC||220||>
|-
|DD||221||=
|-
|DE||222||<
|-
|DF||223||SGN
|-
|E0||224||INT
|-
|E1||225||ABS
|-
|E2||226||FRE
|-
|E3||227||INP
|-
|E4||228||LPOS
|-
|E5||229||POS
|-
|E6||230||SQR
|-
|E7||231||RND
|-
|E8||232||LOG
|-
|E9||233||EXP
|-
|EA||234||COS
|-
|EB||235||SIN
|-
|EC||236||TAN
|-
|ED||237||ATN
|-
|EE||238||PEEK
|-
|EF||239||EOF
|-
|F0||240||LOC
|-
|F1||241||LOF
|-
|F2||242||CINT
|-
|F3||243||CSNG
|-
|F4||244||CDBL
|-
|F5||245||FIX
|-
|F6||246||LEN
|-
|F7||247||STR$
|-
|F8||248||VAL
|-
|F9||249||ASC
|-
|FA||250||CHR$
|-
|FB||251||SPACE$
|-
|FC||252||LEFT$
|-
|FD||253||RIGHT$
|-
|FE||254||MID$
|-
|FF||255||' (QUOTE) || When tokenizing, the single quote character expands to three characters: a colon (3A), the byte for REM (8E), and then FF.
|}

== File Format ==

Tokenised BASIC code is a sequence of tokenized lines. Each tokenized line has the following format:

{|class="wikitable"
|+ Format of a single tokenized line
!Name!!Length!!Description
|-
|PL PH
|2 bytes
|Address in RAM of the next line of BASIC; unsigned 16-bit integer, little-endian. In a file, PL PH is merely a placeholder and the value is never used. In RAM, PL PH == 0 marks the end of the tokenized BASIC listing. (See [[#Details for PL PH]].)
|-
|LL LH
|2 bytes
|Line number; unsigned 16-bit integer, little-endian.
|-
|B0 … Bn
|Any number of bytes
|Depending on the high bit, each byte is either a token or an ASCII character.
|-
|NULL
|1 byte
|NULL byte (<code>0x00</code>) to signal end of line.
|}

Typically, the list of tokenized lines continues until the end of the file. Once a program is loaded into RAM, the end is signaled by two consecutive NULLs in the location where PL PH would normally start the next line.

=== Limits ===

* Line numbers can range from 0 to 65529 (but see variances below).
* Tokenized line length is limited to 256 characters, including the trailing NULL byte. (To do: double check this.)

=== Acceptable variances ===

While Tandy BASIC will never generate such files, it has no problem loading a tokenized program that:

* Has line numbers out of order; the numerical value of a line number is ignored. Program lines are always LISTed and RUN in the order they appear in the tokenized file.
* Contains duplicate line numbers; all lines are kept and run in the order found in the file. (GOTO is complicated as search starts at the next line, not the beginning of the program.)
* Contains a line with zero bytes of data; LIST will show the line number with no text afterward. Does not replace or delete a preexisting line with that number.
* Contains arbitrary bytes for PL PH; the value of PL PH is always recalculated at LOAD time.
* Contains line numbers greater than the nominal maximum of 65529; this can be used to store hidden binary data [https://www.mail-archive.com/m100@lists.bitchin100.com/msg16195.html according to John R. Hogerhuis].

''Sidenote'': while these variances are accepted by actual hardware, emulators may not be able to handle such files. For example, [https://sourceforge.net/projects/virtualt/ Virtual T] (as of version 1.7 in 2022) refuses to load tokenized BASIC programs that have lines out of order, duplicate lines, or bogus values for PL PH.

=== Details for PL PH ===
In a file, the specific values of the 16-bit integer PL PH are ''never'' used. Tandy BASIC always recalculates PL PH before loading a program into RAM. Although not technically part of the file format, one may want to calculate PL PH for compatibility with broken or persnickety emulators.

<div class="toccolours mw-collapsible mw-collapsed">
==== Calculating PL PH ====
<div class="mw-collapsible-content">
Once in RAM, PL PH represents the memory address of the next line. According to John Hogerhuis, the purpose is to quickly skip through lines in order to reach the target of GOTO, GOSUB, and READ. Additionally, the end of a tokenized BASIC listing is marked by the final line's PL PH equal to zero (two NULL bytes).

By default on the Tandy 200, the first token of the first line of the BASIC program in RAM is at address 0xA001.
</div>
</div>

== Format documentation ==

* [https://m100.bitchin100.narkive.com/eP2uSl4J/tokenized-basic-programs-bytes-of-mystery Discussion of byte format on Bitchin 100 Mailing List]
* [http://www.club100.org/library/ups/tokens.do BASIC program to create a table of tokens]
* [https://archive.org/details/HiddenPowersOfTheTrs80Model100/page/n229/mode/2up?view=theater Hidden Powers of the TRS 80 Model 100]. Appendix B lists the BASIC token values for the Model 100. Other than the omission of 255 for a single quote remark, this reference matches what hackerb9 found programmatically for the Tandy 200.
* [http://web.archive.org/web/20220324135218/https://help.ayra.ch/trs80-reference#trs-80-model-100-basic-reference Ayra Model 100 BASIC Reference]

== Other links ==

* [https://github.com/hackerb9/tokenize/ Hackerb9's C program for tokenizing BASIC code]
* [https://sourceforge.net/projects/virtualt/ A Tandy 200 & Model 100/102 emulator]
* [http://bitchin100.com/CloudT/ A TRS-80 Model 100 emulator that runs in a web browser]

[[Category:TRS-80]]
[[Category:Microsoft]]

Tandy 200 BASIC tokenized file

2024-03-10T22:21:41Z

Hackerb9: /* Details for PL PH */ Fill in more details, make section collapsed by default as it will grow longer once an example is added

{{FormatInfo
|subcat=Source code
|subcat2=Tokenized BASIC
|released=1983
}}

'''Tandy 200 BASIC''' was a version of Microsoft BASIC for the Radio Shack Tandy 200 computer. The tokenizations for the TRS-80 Model 100 and Tandy 102 appear to be identical. In fact, while the research that created this page was done on a Tandy 200, experiments using [https://sourceforge.net/projects/virtualt/ emulated systems] indicate that the tokenized BASIC file format is the same across most of the computers related to the Kyocera Kyotronic-85, with the exception being the models made by NEC.

{| class="wikitable" style="margin-left: auto; margin-right: auto; border: none;"
|+ Which of the Kyotronic-85 Sisters use this format
! title="Name of the manufacturer and computer model" | Computer Model
! title="Whether the tokenization is the same as for the Tandy 200" | Uses Tandy 200 Tokenization?
|- style=text-align:center
|Kyocera Kyotronic-85 Olivetti M10 TRS-80 Model 100 Tandy 200 Tandy 102||Yes
|- style=text-align:center
| NEC PC-8201 NEC PC-8201A NEC PC-8300 || No
|}

While the NEC PC-8201/8300 format is not the same, the BASIC it runs (N82 BASIC) is very close and it is expected the tokenization will be similar.

== Tokens ==
All BASIC tokens on the Tandy 200 are a single byte with a value from 128 to 255.

{| class="wikitable"
|+ Table mapping token values to BASIC keywords
! title="The value of the token in hexadecimal" | Hex
! title="The value of the token in decimal" | Dec
! title="The BASIC keyword the token stands for" | Keyword
! title="Miscellany" | Notes
|-
|-
|80||128||END
|-
|81||129||FOR
|-
|82||130||NEXT
|-
|83||131||DATA
|-
|84||132||INPUT
|-
|85||133||DIM
|-
|86||134||READ
|-
|87||135||LET
|-
|88||136||GOTO
|-
|89||137||RUN
|-
|8A||138||IF
|-
|8B||139||RESTORE
|-
|8C||140||GOSUB
|-
|8D||141||RETURN
|-
|8E||142||REM
|-
|8F||143||STOP
|-
|90||144||WIDTH
|-
|91||145||ELSE||When tokenizing, the Tandy 200 always adds a colon (':') before the ELSE token (3A 91). If the user actually writes :ELSE it is tokenized as 3A 3A 91.
|-
|92||146||LINE
|-
|93||147||EDIT
|-
|94||148||ERROR
|-
|95||149||RESUME
|-
|96||150||OUT
|-
|97||151||ON
|-
|98||152||DSKO$
|-
|99||153||OPEN
|-
|9A||154||CLOSE
|-
|9B||155||LOAD||Note that LOADM is simply the token for LOAD followed by an ASCII 'M'.
|-
|9C||156||MERGE
|-QUOTE
|9D||157||FILES
|-
|9E||158||SAVE
|-
|9F||159||LFILES
|-
|A0||160||LPRINT
|-
|A1||161||DEF
|-
|A2||162||POKE
|-
|A3||163||PRINT
|-
|A4||164||CONT
|-
|A5||165||LIST
|-
|A6||166||LLIST
|-
|A7||167||CLEAR
|-
|A8||168||CLOAD
|-
|A9||169||CSAVE
|-
|AA||170||TIME$
|-
|AB||171||DATE$
|-
|AC||172||DAY$
|-
|AD||173||COM
|-
|AE||174||MDM
|-
|AF||175||KEY
|-
|B0||176||CLS
|-
|B1||177||BEEP
|-
|B2||178||SOUND
|-
|B3||179||LCOPY
|-
|B4||180||PSET
|-
|B5||181||PRESET
|-
|B6||182||MOTOR
|-
|B7||183||MAX
|-
|B8||184||POWER
|-
|B9||185||CALL
|-
|BA||186||MENU
|-
|BB||187||IPL
|-
|BC||188||NAME
|-
|BD||189||KILL
|-
|BE||190||SCREEN
|-
|BF||191||NEW
|-
|C0||192||TAB(
|-
|C1||193||TO
|-
|C2||194||USING
|-
|C3||195||VARPTR
|-
|C4||196||ERL
|-
|C5||197||ERR
|-
|C6||198||STRING$
|-
|C7||199||INSTR
|-
|C8||200||DSKI$
|-
|C9||201||INKEY$
|-
|CA||202||CSRLIN
|-
|CB||203||OFF
|-
|CC||204||HIMEM
|-
|CD||205||THEN
|-
|CE||206||NOT
|-
|CF||207||STEP
|-
|D0||208||+
|-
|D1||209||-
|-
|D2||210||*
|-
|D3||211||/
|-
|D4||212||^
|-
|D5||213||AND
|-
|D6||214||OR
|-
|D7||215||XOR
|-
|D8||216||EQV
|-
|D9||217||IMP
|-
|DA||218||MOD
|-
|DB||219||\
|-
|DC||220||>
|-
|DD||221||=
|-
|DE||222||<
|-
|DF||223||SGN
|-
|E0||224||INT
|-
|E1||225||ABS
|-
|E2||226||FRE
|-
|E3||227||INP
|-
|E4||228||LPOS
|-
|E5||229||POS
|-
|E6||230||SQR
|-
|E7||231||RND
|-
|E8||232||LOG
|-
|E9||233||EXP
|-
|EA||234||COS
|-
|EB||235||SIN
|-
|EC||236||TAN
|-
|ED||237||ATN
|-
|EE||238||PEEK
|-
|EF||239||EOF
|-
|F0||240||LOC
|-
|F1||241||LOF
|-
|F2||242||CINT
|-
|F3||243||CSNG
|-
|F4||244||CDBL
|-
|F5||245||FIX
|-
|F6||246||LEN
|-
|F7||247||STR$
|-
|F8||248||VAL
|-
|F9||249||ASC
|-
|FA||250||CHR$
|-
|FB||251||SPACE$
|-
|FC||252||LEFT$
|-
|FD||253||RIGHT$
|-
|FE||254||MID$
|-
|FF||255||' (QUOTE) || When tokenizing, the single quote character expands to three characters: a colon (3A), the byte for REM (8E), and then FF.
|}

== File Format ==

Tokenised BASIC code is a sequence of tokenized lines. Each tokenized line has the following format:

{|class="wikitable"
|+ Format of a single tokenized line
!Name!!Length!!Description
|-
|PL PH
|2 bytes
|Address in RAM of the next line of BASIC; unsigned 16-bit integer, little-endian. In a file, PL PH is merely a placeholder and the value is never used. In RAM, PL PH == 0 marks the end of the tokenized BASIC listing. (See [[#Details for PL PH]].)
|-
|LL LH
|2 bytes
|Line number; unsigned 16-bit integer, little-endian.
|-
|B0 … Bn
|Any number of bytes
|Depending on the high bit, each byte is either a token or an ASCII character.
|-
|NULL
|1 byte
|NULL byte (<code>0x00</code>) to signal end of line.
|}

Typically, the list of tokenized lines continues until the end of the file. Once a program is loaded into RAM, the end is signaled by two consecutive NULLs in the location where PL PH would normally start the next line.

=== Limits ===

* Line numbers can range from 0 to 65529 (but see variances below).
* Tokenized line length is limited to 256 characters, including the trailing NULL byte. (To do: double check this.)

=== Acceptable variances ===

While Tandy BASIC will never generate such files, it has no problem loading a tokenized program that:

* Has line numbers out of order; the numerical value of a line number is ignored. Program lines are always LISTed and RUN in the order they appear in the tokenized file.
* Contains duplicate line numbers; all lines are kept and run in the order found in the file. (GOTO is complicated as search starts at the next line, not the beginning of the program.)
* Contains a line with zero bytes of data; LIST will show the line number with no text afterward. Does not replace or delete a preexisting line with that number.
* Contains arbitrary bytes for PL PH; the value of PL PH is always recalculated at LOAD time.
* Contains line numbers greater than the nominal maximum of 65529; this can be used to store hidden binary data [https://www.mail-archive.com/m100@lists.bitchin100.com/msg16195.html according to John R. Hogerhuis].

''Sidenote'': while these variances are accepted by actual hardware, emulators may not be able to handle such files. For example, [https://sourceforge.net/projects/virtualt/ Virtual T] (as of version 1.7 in 2022) refuses to load tokenized BASIC programs that have lines out of order, duplicate lines, or bogus values for PL PH.

=== Details for PL PH ===
In a file, the specific values of the 16-bit integer PL PH are ''never'' used. Tandy BASIC always recalculates PL PH before loading a program into RAM. Although not technically part of the file format, one may want to calculate PL PH for compatibility with broken or persnickety emulators.

<div class="toccolours mw-collapsible mw-collapsed">
==== Calculating PL PH ====
<div class="mw-collapsible-content">
Once in RAM, PL PH represents the memory address of the next line. According to John Hogerhuis, the purpose is to quickly skip through lines in order to reach the target of GOTO, GOSUB, and READ. Additionally, the end of a tokenized BASIC listing is marked by the final line's PL PH equal to zero (two NULL bytes).

By default on the Tandy 200, the first line of the BASIC program in RAM starts at address 0xA001. However, that can be changed by the HIMEM command.
</div>
</div>

== Format documentation ==

* [https://m100.bitchin100.narkive.com/eP2uSl4J/tokenized-basic-programs-bytes-of-mystery Discussion of byte format on Bitchin 100 Mailing List]
* [http://www.club100.org/library/ups/tokens.do BASIC program to create a table of tokens]
* [https://archive.org/details/HiddenPowersOfTheTrs80Model100/page/n229/mode/2up?view=theater Hidden Powers of the TRS 80 Model 100]. Appendix B lists the BASIC token values for the Model 100. Other than the omission of 255 for a single quote remark, this reference matches what hackerb9 found programmatically for the Tandy 200.
* [http://web.archive.org/web/20220324135218/https://help.ayra.ch/trs80-reference#trs-80-model-100-basic-reference Ayra Model 100 BASIC Reference]

== Other links ==

* [https://github.com/hackerb9/tokenize/ Hackerb9's C program for tokenizing BASIC code]
* [https://sourceforge.net/projects/virtualt/ A Tandy 200 & Model 100/102 emulator]
* [http://bitchin100.com/CloudT/ A TRS-80 Model 100 emulator that runs in a web browser]

[[Category:TRS-80]]
[[Category:Microsoft]]

Tandy 200 BASIC tokenized file

2022-11-05T10:59:02Z

Hackerb9: Add info about other platforms that use the same tokenization format

{{FormatInfo
|subcat=Source code
|subcat2=Tokenized BASIC
|released=1983
}}

'''Tandy 200 BASIC''' was a version of Microsoft BASIC for the Radio Shack Tandy 200 computer. The tokenizations for the TRS-80 Model 100 and Tandy 102 appear to be identical. In fact, while the research that created this page was done on a Tandy 200, experiments using [https://sourceforge.net/projects/virtualt/ emulated systems] indicate that the tokenized BASIC file format is the same across most of the computers related to the Kyocera Kyotronic-85, with the exception being the models made by NEC.

{| class="wikitable" style="margin-left: auto; margin-right: auto; border: none;"
|+ Which of the Kyotronic-85 Sisters use this format
! title="Name of the manufacturer and computer model" | Computer Model
! title="Whether the tokenization is the same as for the Tandy 200" | Uses Tandy 200 Tokenization?
|- style=text-align:center
|Kyocera Kyotronic-85 Olivetti M10 TRS-80 Model 100 Tandy 200 Tandy 102||Yes
|- style=text-align:center
| NEC PC-8201 NEC PC-8201A NEC PC-8300 || No
|}

While the NEC PC-8201/8300 format is not the same, the BASIC it runs (N82 BASIC) is very close and it is expected the tokenization will be similar.

== Tokens ==
All BASIC tokens on the Tandy 200 are a single byte with a value from 128 to 255.

{| class="wikitable"
|+ Table mapping token values to BASIC keywords
! title="The value of the token in hexadecimal" | Hex
! title="The value of the token in decimal" | Dec
! title="The BASIC keyword the token stands for" | Keyword
! title="Miscellany" | Notes
|-
|-
|80||128||END
|-
|81||129||FOR
|-
|82||130||NEXT
|-
|83||131||DATA
|-
|84||132||INPUT
|-
|85||133||DIM
|-
|86||134||READ
|-
|87||135||LET
|-
|88||136||GOTO
|-
|89||137||RUN
|-
|8A||138||IF
|-
|8B||139||RESTORE
|-
|8C||140||GOSUB
|-
|8D||141||RETURN
|-
|8E||142||REM
|-
|8F||143||STOP
|-
|90||144||WIDTH
|-
|91||145||ELSE||When tokenizing, the Tandy 200 always adds a colon (':') before the ELSE token (3A 91). If the user actually writes :ELSE it is tokenized as 3A 3A 91.
|-
|92||146||LINE
|-
|93||147||EDIT
|-
|94||148||ERROR
|-
|95||149||RESUME
|-
|96||150||OUT
|-
|97||151||ON
|-
|98||152||DSKO$
|-
|99||153||OPEN
|-
|9A||154||CLOSE
|-
|9B||155||LOAD||Note that LOADM is simply the token for LOAD followed by an ASCII 'M'.
|-
|9C||156||MERGE
|-QUOTE
|9D||157||FILES
|-
|9E||158||SAVE
|-
|9F||159||LFILES
|-
|A0||160||LPRINT
|-
|A1||161||DEF
|-
|A2||162||POKE
|-
|A3||163||PRINT
|-
|A4||164||CONT
|-
|A5||165||LIST
|-
|A6||166||LLIST
|-
|A7||167||CLEAR
|-
|A8||168||CLOAD
|-
|A9||169||CSAVE
|-
|AA||170||TIME$
|-
|AB||171||DATE$
|-
|AC||172||DAY$
|-
|AD||173||COM
|-
|AE||174||MDM
|-
|AF||175||KEY
|-
|B0||176||CLS
|-
|B1||177||BEEP
|-
|B2||178||SOUND
|-
|B3||179||LCOPY
|-
|B4||180||PSET
|-
|B5||181||PRESET
|-
|B6||182||MOTOR
|-
|B7||183||MAX
|-
|B8||184||POWER
|-
|B9||185||CALL
|-
|BA||186||MENU
|-
|BB||187||IPL
|-
|BC||188||NAME
|-
|BD||189||KILL
|-
|BE||190||SCREEN
|-
|BF||191||NEW
|-
|C0||192||TAB(
|-
|C1||193||TO
|-
|C2||194||USING
|-
|C3||195||VARPTR
|-
|C4||196||ERL
|-
|C5||197||ERR
|-
|C6||198||STRING$
|-
|C7||199||INSTR
|-
|C8||200||DSKI$
|-
|C9||201||INKEY$
|-
|CA||202||CSRLIN
|-
|CB||203||OFF
|-
|CC||204||HIMEM
|-
|CD||205||THEN
|-
|CE||206||NOT
|-
|CF||207||STEP
|-
|D0||208||+
|-
|D1||209||-
|-
|D2||210||*
|-
|D3||211||/
|-
|D4||212||^
|-
|D5||213||AND
|-
|D6||214||OR
|-
|D7||215||XOR
|-
|D8||216||EQV
|-
|D9||217||IMP
|-
|DA||218||MOD
|-
|DB||219||\
|-
|DC||220||>
|-
|DD||221||=
|-
|DE||222||<
|-
|DF||223||SGN
|-
|E0||224||INT
|-
|E1||225||ABS
|-
|E2||226||FRE
|-
|E3||227||INP
|-
|E4||228||LPOS
|-
|E5||229||POS
|-
|E6||230||SQR
|-
|E7||231||RND
|-
|E8||232||LOG
|-
|E9||233||EXP
|-
|EA||234||COS
|-
|EB||235||SIN
|-
|EC||236||TAN
|-
|ED||237||ATN
|-
|EE||238||PEEK
|-
|EF||239||EOF
|-
|F0||240||LOC
|-
|F1||241||LOF
|-
|F2||242||CINT
|-
|F3||243||CSNG
|-
|F4||244||CDBL
|-
|F5||245||FIX
|-
|F6||246||LEN
|-
|F7||247||STR$
|-
|F8||248||VAL
|-
|F9||249||ASC
|-
|FA||250||CHR$
|-
|FB||251||SPACE$
|-
|FC||252||LEFT$
|-
|FD||253||RIGHT$
|-
|FE||254||MID$
|-
|FF||255||' (QUOTE) || When tokenizing, the single quote character expands to three characters: a colon (3A), the byte for REM (8E), and then FF.
|}

== File Format ==

Tokenised BASIC code is a sequence of tokenized lines. Each tokenized line has the following format:

{|class="wikitable"
|+ Format of a single tokenized line
!Name!!Length!!Description
|-
|PL PH
|2 bytes
|Address in RAM of the next line of BASIC; unsigned 16-bit integer, little-endian. In a file, PL PH is merely a placeholder and the value is never used. In RAM, PL PH == 0 marks the end of the tokenized BASIC listing. (See [[#Details for PL PH]].)
|-
|LL LH
|2 bytes
|Line number; unsigned 16-bit integer, little-endian.
|-
|B0 … Bn
|Any number of bytes
|Depending on the high bit, each byte is either a token or an ASCII character.
|-
|NULL
|1 byte
|NULL byte (<code>0x00</code>) to signal end of line.
|}

Typically, the list of tokenized lines continues until the end of the file. Once a program is loaded into RAM, the end is signaled by two consecutive NULLs in the location where PL PH would normally start the next line.

=== Limits ===

* Line numbers can range from 0 to 65529 (but see variances below).
* Tokenized line length is limited to 256 characters, including the trailing NULL byte. (To do: double check this.)

=== Acceptable variances ===

While Tandy BASIC will never generate such files, it has no problem loading a tokenized program that:

* Has line numbers out of order; the numerical value of a line number is ignored. Program lines are always LISTed and RUN in the order they appear in the tokenized file.
* Contains duplicate line numbers; all lines are kept and run in the order found in the file. (GOTO is complicated as search starts at the next line, not the beginning of the program.)
* Contains a line with zero bytes of data; LIST will show the line number with no text afterward. Does not replace or delete a preexisting line with that number.
* Contains arbitrary bytes for PL PH; the value of PL PH is always recalculated at LOAD time.
* Contains line numbers greater than the nominal maximum of 65529; this can be used to store hidden binary data [https://www.mail-archive.com/m100@lists.bitchin100.com/msg16195.html according to John R. Hogerhuis].

''Sidenote'': while these variances are accepted by actual hardware, emulators may not be able to handle such files. For example, [https://sourceforge.net/projects/virtualt/ Virtual T] (as of version 1.7 in 2022) refuses to load tokenized BASIC programs that have lines out of order, duplicate lines, or bogus values for PL PH.

=== Details for PL PH ===

In a file, the specific values of the 16-bit integer PL PH are never used. During program load, Tandy BASIC recalculates PL PH before storing it into RAM. Once in memory, PL PH is used to skip to the line required for GOTO, GOSUB, and READ, thus speeding up the program. Additionally, the end of a tokenized BASIC listing in RAM is marked by PL PH equal to zero (two NULL bytes).

(Stub. This is where Tandy BASIC's calculation of PL PH values will go.)

== Format documentation ==

* [https://m100.bitchin100.narkive.com/eP2uSl4J/tokenized-basic-programs-bytes-of-mystery Discussion of byte format on Bitchin 100 Mailing List]
* [http://www.club100.org/library/ups/tokens.do BASIC program to create a table of tokens]
* [https://archive.org/details/HiddenPowersOfTheTrs80Model100/page/n229/mode/2up?view=theater Hidden Powers of the TRS 80 Model 100]. Appendix B lists the BASIC token values for the Model 100. Other than the omission of 255 for a single quote remark, this reference matches what hackerb9 found programmatically for the Tandy 200.
* [http://web.archive.org/web/20220324135218/https://help.ayra.ch/trs80-reference#trs-80-model-100-basic-reference Ayra Model 100 BASIC Reference]

== Other links ==

* [https://github.com/hackerb9/tokenize/ Hackerb9's C program for tokenizing BASIC code]
* [https://sourceforge.net/projects/virtualt/ A Tandy 200 & Model 100/102 emulator]
* [http://bitchin100.com/CloudT/ A TRS-80 Model 100 emulator that runs in a web browser]

[[Category:Microsoft]]

Tandy 200 BASIC tokenized file

2022-10-06T03:36:50Z

Hackerb9: /* Other links */

{{FormatInfo
|subcat=Source code
|subcat2=Tokenized BASIC
|released=1984
}}

'''Tandy 200 BASIC''' was a version of Microsoft BASIC for the Radio Shack Tandy 200 computer. The tokenizations for the TRS-80 Model 100 and Tandy 102 appear to be identical. The NEC PC-8201/8300 format is not the same, but should be similar.

== Tokens ==
All BASIC tokens on the Tandy 200 are a single byte with a value from 128 to 255.

{| class="wikitable"
|+ Table mapping token values to BASIC keywords
! title="The value of the token in hexadecimal" | Hex
! title="The value of the token in decimal" | Dec
! title="The BASIC keyword the token stands for" | Keyword
! title="Miscellany" | Notes
|-
|-
|80||128||END
|-
|81||129||FOR
|-
|82||130||NEXT
|-
|83||131||DATA
|-
|84||132||INPUT
|-
|85||133||DIM
|-
|86||134||READ
|-
|87||135||LET
|-
|88||136||GOTO
|-
|89||137||RUN
|-
|8A||138||IF
|-
|8B||139||RESTORE
|-
|8C||140||GOSUB
|-
|8D||141||RETURN
|-
|8E||142||REM
|-
|8F||143||STOP
|-
|90||144||WIDTH
|-
|91||145||ELSE||When tokenizing, the Tandy 200 always adds a colon (':') before the ELSE token (3A 91). If the user actually writes :ELSE it is tokenized as 3A 3A 91.
|-
|92||146||LINE
|-
|93||147||EDIT
|-
|94||148||ERROR
|-
|95||149||RESUME
|-
|96||150||OUT
|-
|97||151||ON
|-
|98||152||DSKO$
|-
|99||153||OPEN
|-
|9A||154||CLOSE
|-
|9B||155||LOAD||Note that LOADM is simply the token for LOAD followed by an ASCII 'M'.
|-
|9C||156||MERGE
|-QUOTE
|9D||157||FILES
|-
|9E||158||SAVE
|-
|9F||159||LFILES
|-
|A0||160||LPRINT
|-
|A1||161||DEF
|-
|A2||162||POKE
|-
|A3||163||PRINT
|-
|A4||164||CONT
|-
|A5||165||LIST
|-
|A6||166||LLIST
|-
|A7||167||CLEAR
|-
|A8||168||CLOAD
|-
|A9||169||CSAVE
|-
|AA||170||TIME$
|-
|AB||171||DATE$
|-
|AC||172||DAY$
|-
|AD||173||COM
|-
|AE||174||MDM
|-
|AF||175||KEY
|-
|B0||176||CLS
|-
|B1||177||BEEP
|-
|B2||178||SOUND
|-
|B3||179||LCOPY
|-
|B4||180||PSET
|-
|B5||181||PRESET
|-
|B6||182||MOTOR
|-
|B7||183||MAX
|-
|B8||184||POWER
|-
|B9||185||CALL
|-
|BA||186||MENU
|-
|BB||187||IPL
|-
|BC||188||NAME
|-
|BD||189||KILL
|-
|BE||190||SCREEN
|-
|BF||191||NEW
|-
|C0||192||TAB(
|-
|C1||193||TO
|-
|C2||194||USING
|-
|C3||195||VARPTR
|-
|C4||196||ERL
|-
|C5||197||ERR
|-
|C6||198||STRING$
|-
|C7||199||INSTR
|-
|C8||200||DSKI$
|-
|C9||201||INKEY$
|-
|CA||202||CSRLIN
|-
|CB||203||OFF
|-
|CC||204||HIMEM
|-
|CD||205||THEN
|-
|CE||206||NOT
|-
|CF||207||STEP
|-
|D0||208||+
|-
|D1||209||-
|-
|D2||210||*
|-
|D3||211||/
|-
|D4||212||^
|-
|D5||213||AND
|-
|D6||214||OR
|-
|D7||215||XOR
|-
|D8||216||EQV
|-
|D9||217||IMP
|-
|DA||218||MOD
|-
|DB||219||\
|-
|DC||220||>
|-
|DD||221||=
|-
|DE||222||<
|-
|DF||223||SGN
|-
|E0||224||INT
|-
|E1||225||ABS
|-
|E2||226||FRE
|-
|E3||227||INP
|-
|E4||228||LPOS
|-
|E5||229||POS
|-
|E6||230||SQR
|-
|E7||231||RND
|-
|E8||232||LOG
|-
|E9||233||EXP
|-
|EA||234||COS
|-
|EB||235||SIN
|-
|EC||236||TAN
|-
|ED||237||ATN
|-
|EE||238||PEEK
|-
|EF||239||EOF
|-
|F0||240||LOC
|-
|F1||241||LOF
|-
|F2||242||CINT
|-
|F3||243||CSNG
|-
|F4||244||CDBL
|-
|F5||245||FIX
|-
|F6||246||LEN
|-
|F7||247||STR$
|-
|F8||248||VAL
|-
|F9||249||ASC
|-
|FA||250||CHR$
|-
|FB||251||SPACE$
|-
|FC||252||LEFT$
|-
|FD||253||RIGHT$
|-
|FE||254||MID$
|-
|FF||255||' (QUOTE) || When tokenizing, the single quote character expands to three characters: a colon (3A), the byte for REM (8E), and then FF.
|}

== File Format ==

Tokenised BASIC code is a sequence of tokenized lines. Each tokenized line has the following format:

{|class="wikitable"
|+ Format of a single tokenized line
!Name!!Length!!Description
|-
|PL PH
|2 bytes
|Address in RAM of the next line of BASIC; unsigned 16-bit integer, little-endian. In a file, PL PH is merely a placeholder and the value is never used. In RAM, PL PH == 0 marks the end of the tokenized BASIC listing. (See [[#Details for PL PH]].)
|-
|LL LH
|2 bytes
|Line number; unsigned 16-bit integer, little-endian.
|-
|B0 … Bn
|Any number of bytes
|Depending on the high bit, each byte is either a token or an ASCII character.
|-
|NULL
|1 byte
|NULL byte (<code>0x00</code>) to signal end of line.
|}

Typically, the list of tokenized lines continues until the end of the file. Once a program is loaded into RAM, the end is signaled by two consecutive NULLs in the location where PL PH would normally start the next line.

=== Limits ===

* Line numbers can range from 0 to 65529 (but see variances below).
* Tokenized line length is limited to 256 characters, including the trailing NULL byte. (To do: double check this.)

=== Acceptable variances ===

While Tandy BASIC will never generate such files, it has no problem loading a tokenized program that:

* Has line numbers out of order; the numerical value of a line number is ignored. Program lines are always LISTed and RUN in the order they appear in the tokenized file.
* Contains duplicate line numbers; all lines are kept and run in the order found in the file. (GOTO is complicated as search starts at the next line, not the beginning of the program.)
* Contains a line with zero bytes of data; LIST will show the line number with no text afterward. Does not replace or delete a preexisting line with that number.
* Contains arbitrary bytes for PL PH; the value of PL PH is always recalculated at LOAD time.
* Contains line numbers greater than the nominal maximum of 65529; this can be used to store hidden binary data [https://www.mail-archive.com/m100@lists.bitchin100.com/msg16195.html according to John R. Hogerhuis].

''Sidenote'': while these variances are accepted by actual hardware, emulators may not be able to handle such files. For example, [https://sourceforge.net/projects/virtualt/ Virtual T] (as of version 1.7 in 2022) refuses to load tokenized BASIC programs that have lines out of order, duplicate lines, or bogus values for PL PH.

=== Details for PL PH ===

In a file, the specific values of the 16-bit integer PL PH are never used. During program load, Tandy BASIC recalculates PL PH before storing it into RAM. Once in memory, PL PH is used to skip to the line required for GOTO, GOSUB, and READ, thus speeding up the program. Additionally, the end of a tokenized BASIC listing in RAM is marked by PL PH equal to zero (two NULL bytes).

(Stub. This is where Tandy BASIC's calculation of PL PH values will go.)

== Format documentation ==

* [https://m100.bitchin100.narkive.com/eP2uSl4J/tokenized-basic-programs-bytes-of-mystery Discussion of byte format on Bitchin 100 Mailing List]
* [http://www.club100.org/library/ups/tokens.do BASIC program to create a table of tokens]
* [https://archive.org/details/HiddenPowersOfTheTrs80Model100/page/n229/mode/2up?view=theater Hidden Powers of the TRS 80 Model 100]. Appendix B lists the BASIC token values for the Model 100. Other than the omission of 255 for a single quote remark, this reference matches what hackerb9 found programmatically for the Tandy 200.
* [http://web.archive.org/web/20220324135218/https://help.ayra.ch/trs80-reference#trs-80-model-100-basic-reference Ayra Model 100 BASIC Reference]

== Other links ==

* [https://github.com/hackerb9/tokenize/ Hackerb9's C program for tokenizing BASIC code]
* [https://sourceforge.net/projects/virtualt/ A Tandy 200 & Model 100/102 emulator]
* [http://bitchin100.com/CloudT/ A TRS-80 Model 100 emulator that runs in a web browser]

[[Category:Microsoft]]

Tandy 200 BASIC tokenized file

2022-10-06T00:24:45Z

Hackerb9: /* Details for PL PH */ grammar

{{FormatInfo
|subcat=Source code
|subcat2=Tokenized BASIC
|released=1984
}}

'''Tandy 200 BASIC''' was a version of Microsoft BASIC for the Radio Shack Tandy 200 computer. The tokenizations for the TRS-80 Model 100 and Tandy 102 appear to be identical. The NEC PC-8201/8300 format is not the same, but should be similar.

== Tokens ==
All BASIC tokens on the Tandy 200 are a single byte with a value from 128 to 255.

{| class="wikitable"
|+ Table mapping token values to BASIC keywords
! title="The value of the token in hexadecimal" | Hex
! title="The value of the token in decimal" | Dec
! title="The BASIC keyword the token stands for" | Keyword
! title="Miscellany" | Notes
|-
|-
|80||128||END
|-
|81||129||FOR
|-
|82||130||NEXT
|-
|83||131||DATA
|-
|84||132||INPUT
|-
|85||133||DIM
|-
|86||134||READ
|-
|87||135||LET
|-
|88||136||GOTO
|-
|89||137||RUN
|-
|8A||138||IF
|-
|8B||139||RESTORE
|-
|8C||140||GOSUB
|-
|8D||141||RETURN
|-
|8E||142||REM
|-
|8F||143||STOP
|-
|90||144||WIDTH
|-
|91||145||ELSE||When tokenizing, the Tandy 200 always adds a colon (':') before the ELSE token (3A 91). If the user actually writes :ELSE it is tokenized as 3A 3A 91.
|-
|92||146||LINE
|-
|93||147||EDIT
|-
|94||148||ERROR
|-
|95||149||RESUME
|-
|96||150||OUT
|-
|97||151||ON
|-
|98||152||DSKO$
|-
|99||153||OPEN
|-
|9A||154||CLOSE
|-
|9B||155||LOAD||Note that LOADM is simply the token for LOAD followed by an ASCII 'M'.
|-
|9C||156||MERGE
|-QUOTE
|9D||157||FILES
|-
|9E||158||SAVE
|-
|9F||159||LFILES
|-
|A0||160||LPRINT
|-
|A1||161||DEF
|-
|A2||162||POKE
|-
|A3||163||PRINT
|-
|A4||164||CONT
|-
|A5||165||LIST
|-
|A6||166||LLIST
|-
|A7||167||CLEAR
|-
|A8||168||CLOAD
|-
|A9||169||CSAVE
|-
|AA||170||TIME$
|-
|AB||171||DATE$
|-
|AC||172||DAY$
|-
|AD||173||COM
|-
|AE||174||MDM
|-
|AF||175||KEY
|-
|B0||176||CLS
|-
|B1||177||BEEP
|-
|B2||178||SOUND
|-
|B3||179||LCOPY
|-
|B4||180||PSET
|-
|B5||181||PRESET
|-
|B6||182||MOTOR
|-
|B7||183||MAX
|-
|B8||184||POWER
|-
|B9||185||CALL
|-
|BA||186||MENU
|-
|BB||187||IPL
|-
|BC||188||NAME
|-
|BD||189||KILL
|-
|BE||190||SCREEN
|-
|BF||191||NEW
|-
|C0||192||TAB(
|-
|C1||193||TO
|-
|C2||194||USING
|-
|C3||195||VARPTR
|-
|C4||196||ERL
|-
|C5||197||ERR
|-
|C6||198||STRING$
|-
|C7||199||INSTR
|-
|C8||200||DSKI$
|-
|C9||201||INKEY$
|-
|CA||202||CSRLIN
|-
|CB||203||OFF
|-
|CC||204||HIMEM
|-
|CD||205||THEN
|-
|CE||206||NOT
|-
|CF||207||STEP
|-
|D0||208||+
|-
|D1||209||-
|-
|D2||210||*
|-
|D3||211||/
|-
|D4||212||^
|-
|D5||213||AND
|-
|D6||214||OR
|-
|D7||215||XOR
|-
|D8||216||EQV
|-
|D9||217||IMP
|-
|DA||218||MOD
|-
|DB||219||\
|-
|DC||220||>
|-
|DD||221||=
|-
|DE||222||<
|-
|DF||223||SGN
|-
|E0||224||INT
|-
|E1||225||ABS
|-
|E2||226||FRE
|-
|E3||227||INP
|-
|E4||228||LPOS
|-
|E5||229||POS
|-
|E6||230||SQR
|-
|E7||231||RND
|-
|E8||232||LOG
|-
|E9||233||EXP
|-
|EA||234||COS
|-
|EB||235||SIN
|-
|EC||236||TAN
|-
|ED||237||ATN
|-
|EE||238||PEEK
|-
|EF||239||EOF
|-
|F0||240||LOC
|-
|F1||241||LOF
|-
|F2||242||CINT
|-
|F3||243||CSNG
|-
|F4||244||CDBL
|-
|F5||245||FIX
|-
|F6||246||LEN
|-
|F7||247||STR$
|-
|F8||248||VAL
|-
|F9||249||ASC
|-
|FA||250||CHR$
|-
|FB||251||SPACE$
|-
|FC||252||LEFT$
|-
|FD||253||RIGHT$
|-
|FE||254||MID$
|-
|FF||255||' (QUOTE) || When tokenizing, the single quote character expands to three characters: a colon (3A), the byte for REM (8E), and then FF.
|}

== File Format ==

Tokenised BASIC code is a sequence of tokenized lines. Each tokenized line has the following format:

{|class="wikitable"
|+ Format of a single tokenized line
!Name!!Length!!Description
|-
|PL PH
|2 bytes
|Address in RAM of the next line of BASIC; unsigned 16-bit integer, little-endian. In a file, PL PH is merely a placeholder and the value is never used. In RAM, PL PH == 0 marks the end of the tokenized BASIC listing. (See [[#Details for PL PH]].)
|-
|LL LH
|2 bytes
|Line number; unsigned 16-bit integer, little-endian.
|-
|B0 … Bn
|Any number of bytes
|Depending on the high bit, each byte is either a token or an ASCII character.
|-
|NULL
|1 byte
|NULL byte (<code>0x00</code>) to signal end of line.
|}

Typically, the list of tokenized lines continues until the end of the file. Once a program is loaded into RAM, the end is signaled by two consecutive NULLs in the location where PL PH would normally start the next line.

=== Limits ===

* Line numbers can range from 0 to 65529 (but see variances below).
* Tokenized line length is limited to 256 characters, including the trailing NULL byte. (To do: double check this.)

=== Acceptable variances ===

While Tandy BASIC will never generate such files, it has no problem loading a tokenized program that:

* Has line numbers out of order; the numerical value of a line number is ignored. Program lines are always LISTed and RUN in the order they appear in the tokenized file.
* Contains duplicate line numbers; all lines are kept and run in the order found in the file. (GOTO is complicated as search starts at the next line, not the beginning of the program.)
* Contains a line with zero bytes of data; LIST will show the line number with no text afterward. Does not replace or delete a preexisting line with that number.
* Contains arbitrary bytes for PL PH; the value of PL PH is always recalculated at LOAD time.
* Contains line numbers greater than the nominal maximum of 65529; this can be used to store hidden binary data [https://www.mail-archive.com/m100@lists.bitchin100.com/msg16195.html according to John R. Hogerhuis].

''Sidenote'': while these variances are accepted by actual hardware, emulators may not be able to handle such files. For example, [https://sourceforge.net/projects/virtualt/ Virtual T] (as of version 1.7 in 2022) refuses to load tokenized BASIC programs that have lines out of order, duplicate lines, or bogus values for PL PH.

=== Details for PL PH ===

In a file, the specific values of the 16-bit integer PL PH are never used. During program load, Tandy BASIC recalculates PL PH before storing it into RAM. Once in memory, PL PH is used to skip to the line required for GOTO, GOSUB, and READ, thus speeding up the program. Additionally, the end of a tokenized BASIC listing in RAM is marked by PL PH equal to zero (two NULL bytes).

(Stub. This is where Tandy BASIC's calculation of PL PH values will go.)

== Format documentation ==

* [https://m100.bitchin100.narkive.com/eP2uSl4J/tokenized-basic-programs-bytes-of-mystery Discussion of byte format on Bitchin 100 Mailing List]
* [http://www.club100.org/library/ups/tokens.do BASIC program to create a table of tokens]
* [https://archive.org/details/HiddenPowersOfTheTrs80Model100/page/n229/mode/2up?view=theater Hidden Powers of the TRS 80 Model 100]. Appendix B lists the BASIC token values for the Model 100. Other than the omission of 255 for a single quote remark, this reference matches what hackerb9 found programmatically for the Tandy 200.
* [http://web.archive.org/web/20220324135218/https://help.ayra.ch/trs80-reference#trs-80-model-100-basic-reference Ayra Model 100 BASIC Reference]

== Other links ==

* [https://github.com/hackerb9/tokenize/ Hackerb9's simple program for tokenizing BASIC code]
* [https://sourceforge.net/projects/virtualt/ A Tandy 200 & Model 100/102 emulator]
* [http://bitchin100.com/CloudT/ A TRS-80 Model 100 emulator that runs in a web browser]

[[Category:Microsoft]]

Tandy 200 BASIC tokenized file

2022-10-06T00:23:56Z

Hackerb9: Fix up description of PL PH to separate a program in a file versus a program in RAM

{{FormatInfo
|subcat=Source code
|subcat2=Tokenized BASIC
|released=1984
}}

'''Tandy 200 BASIC''' was a version of Microsoft BASIC for the Radio Shack Tandy 200 computer. The tokenizations for the TRS-80 Model 100 and Tandy 102 appear to be identical. The NEC PC-8201/8300 format is not the same, but should be similar.

== Tokens ==
All BASIC tokens on the Tandy 200 are a single byte with a value from 128 to 255.

{| class="wikitable"
|+ Table mapping token values to BASIC keywords
! title="The value of the token in hexadecimal" | Hex
! title="The value of the token in decimal" | Dec
! title="The BASIC keyword the token stands for" | Keyword
! title="Miscellany" | Notes
|-
|-
|80||128||END
|-
|81||129||FOR
|-
|82||130||NEXT
|-
|83||131||DATA
|-
|84||132||INPUT
|-
|85||133||DIM
|-
|86||134||READ
|-
|87||135||LET
|-
|88||136||GOTO
|-
|89||137||RUN
|-
|8A||138||IF
|-
|8B||139||RESTORE
|-
|8C||140||GOSUB
|-
|8D||141||RETURN
|-
|8E||142||REM
|-
|8F||143||STOP
|-
|90||144||WIDTH
|-
|91||145||ELSE||When tokenizing, the Tandy 200 always adds a colon (':') before the ELSE token (3A 91). If the user actually writes :ELSE it is tokenized as 3A 3A 91.
|-
|92||146||LINE
|-
|93||147||EDIT
|-
|94||148||ERROR
|-
|95||149||RESUME
|-
|96||150||OUT
|-
|97||151||ON
|-
|98||152||DSKO$
|-
|99||153||OPEN
|-
|9A||154||CLOSE
|-
|9B||155||LOAD||Note that LOADM is simply the token for LOAD followed by an ASCII 'M'.
|-
|9C||156||MERGE
|-QUOTE
|9D||157||FILES
|-
|9E||158||SAVE
|-
|9F||159||LFILES
|-
|A0||160||LPRINT
|-
|A1||161||DEF
|-
|A2||162||POKE
|-
|A3||163||PRINT
|-
|A4||164||CONT
|-
|A5||165||LIST
|-
|A6||166||LLIST
|-
|A7||167||CLEAR
|-
|A8||168||CLOAD
|-
|A9||169||CSAVE
|-
|AA||170||TIME$
|-
|AB||171||DATE$
|-
|AC||172||DAY$
|-
|AD||173||COM
|-
|AE||174||MDM
|-
|AF||175||KEY
|-
|B0||176||CLS
|-
|B1||177||BEEP
|-
|B2||178||SOUND
|-
|B3||179||LCOPY
|-
|B4||180||PSET
|-
|B5||181||PRESET
|-
|B6||182||MOTOR
|-
|B7||183||MAX
|-
|B8||184||POWER
|-
|B9||185||CALL
|-
|BA||186||MENU
|-
|BB||187||IPL
|-
|BC||188||NAME
|-
|BD||189||KILL
|-
|BE||190||SCREEN
|-
|BF||191||NEW
|-
|C0||192||TAB(
|-
|C1||193||TO
|-
|C2||194||USING
|-
|C3||195||VARPTR
|-
|C4||196||ERL
|-
|C5||197||ERR
|-
|C6||198||STRING$
|-
|C7||199||INSTR
|-
|C8||200||DSKI$
|-
|C9||201||INKEY$
|-
|CA||202||CSRLIN
|-
|CB||203||OFF
|-
|CC||204||HIMEM
|-
|CD||205||THEN
|-
|CE||206||NOT
|-
|CF||207||STEP
|-
|D0||208||+
|-
|D1||209||-
|-
|D2||210||*
|-
|D3||211||/
|-
|D4||212||^
|-
|D5||213||AND
|-
|D6||214||OR
|-
|D7||215||XOR
|-
|D8||216||EQV
|-
|D9||217||IMP
|-
|DA||218||MOD
|-
|DB||219||\
|-
|DC||220||>
|-
|DD||221||=
|-
|DE||222||<
|-
|DF||223||SGN
|-
|E0||224||INT
|-
|E1||225||ABS
|-
|E2||226||FRE
|-
|E3||227||INP
|-
|E4||228||LPOS
|-
|E5||229||POS
|-
|E6||230||SQR
|-
|E7||231||RND
|-
|E8||232||LOG
|-
|E9||233||EXP
|-
|EA||234||COS
|-
|EB||235||SIN
|-
|EC||236||TAN
|-
|ED||237||ATN
|-
|EE||238||PEEK
|-
|EF||239||EOF
|-
|F0||240||LOC
|-
|F1||241||LOF
|-
|F2||242||CINT
|-
|F3||243||CSNG
|-
|F4||244||CDBL
|-
|F5||245||FIX
|-
|F6||246||LEN
|-
|F7||247||STR$
|-
|F8||248||VAL
|-
|F9||249||ASC
|-
|FA||250||CHR$
|-
|FB||251||SPACE$
|-
|FC||252||LEFT$
|-
|FD||253||RIGHT$
|-
|FE||254||MID$
|-
|FF||255||' (QUOTE) || When tokenizing, the single quote character expands to three characters: a colon (3A), the byte for REM (8E), and then FF.
|}

== File Format ==

Tokenised BASIC code is a sequence of tokenized lines. Each tokenized line has the following format:

{|class="wikitable"
|+ Format of a single tokenized line
!Name!!Length!!Description
|-
|PL PH
|2 bytes
|Address in RAM of the next line of BASIC; unsigned 16-bit integer, little-endian. In a file, PL PH is merely a placeholder and the value is never used. In RAM, PL PH == 0 marks the end of the tokenized BASIC listing. (See [[#Details for PL PH]].)
|-
|LL LH
|2 bytes
|Line number; unsigned 16-bit integer, little-endian.
|-
|B0 … Bn
|Any number of bytes
|Depending on the high bit, each byte is either a token or an ASCII character.
|-
|NULL
|1 byte
|NULL byte (<code>0x00</code>) to signal end of line.
|}

Typically, the list of tokenized lines continues until the end of the file. Once a program is loaded into RAM, the end is signaled by two consecutive NULLs in the location where PL PH would normally start the next line.

=== Limits ===

* Line numbers can range from 0 to 65529 (but see variances below).
* Tokenized line length is limited to 256 characters, including the trailing NULL byte. (To do: double check this.)

=== Acceptable variances ===

While Tandy BASIC will never generate such files, it has no problem loading a tokenized program that:

* Has line numbers out of order; the numerical value of a line number is ignored. Program lines are always LISTed and RUN in the order they appear in the tokenized file.
* Contains duplicate line numbers; all lines are kept and run in the order found in the file. (GOTO is complicated as search starts at the next line, not the beginning of the program.)
* Contains a line with zero bytes of data; LIST will show the line number with no text afterward. Does not replace or delete a preexisting line with that number.
* Contains arbitrary bytes for PL PH; the value of PL PH is always recalculated at LOAD time.
* Contains line numbers greater than the nominal maximum of 65529; this can be used to store hidden binary data [https://www.mail-archive.com/m100@lists.bitchin100.com/msg16195.html according to John R. Hogerhuis].

''Sidenote'': while these variances are accepted by actual hardware, emulators may not be able to handle such files. For example, [https://sourceforge.net/projects/virtualt/ Virtual T] (as of version 1.7 in 2022) refuses to load tokenized BASIC programs that have lines out of order, duplicate lines, or bogus values for PL PH.

=== Details for PL PH ===

In a file, the specific values of the 16-bit integer PL PH in a file are never used. During program load, Tandy BASIC recalculates PL PH before storing it into RAM. Once in memory, PL PH is used to skip to the line required for GOTO, GOSUB, and READ, thus speeding up the program. Additionally, the end of a tokenized BASIC listing in RAM is marked by PL PH equal to zero (two NULL bytes).

(Stub. This is where Tandy BASIC's calculation of PL PH values will go.)

== Format documentation ==

* [https://m100.bitchin100.narkive.com/eP2uSl4J/tokenized-basic-programs-bytes-of-mystery Discussion of byte format on Bitchin 100 Mailing List]
* [http://www.club100.org/library/ups/tokens.do BASIC program to create a table of tokens]
* [https://archive.org/details/HiddenPowersOfTheTrs80Model100/page/n229/mode/2up?view=theater Hidden Powers of the TRS 80 Model 100]. Appendix B lists the BASIC token values for the Model 100. Other than the omission of 255 for a single quote remark, this reference matches what hackerb9 found programmatically for the Tandy 200.
* [http://web.archive.org/web/20220324135218/https://help.ayra.ch/trs80-reference#trs-80-model-100-basic-reference Ayra Model 100 BASIC Reference]

== Other links ==

* [https://github.com/hackerb9/tokenize/ Hackerb9's simple program for tokenizing BASIC code]
* [https://sourceforge.net/projects/virtualt/ A Tandy 200 & Model 100/102 emulator]
* [http://bitchin100.com/CloudT/ A TRS-80 Model 100 emulator that runs in a web browser]

[[Category:Microsoft]]

Tandy 200 BASIC tokenized file

2022-10-05T21:43:10Z

Hackerb9: /* Acceptable variances */ Add note about zero bytes of data

{{FormatInfo
|subcat=Source code
|subcat2=Tokenized BASIC
|released=1984
}}

'''Tandy 200 BASIC''' was a version of Microsoft BASIC for the Radio Shack Tandy 200 computer. The tokenizations for the TRS-80 Model 100 and Tandy 102 appear to be identical. The NEC PC-8201/8300 format is not the same, but should be similar.

== Tokens ==
All BASIC tokens on the Tandy 200 are a single byte with a value from 128 to 255.

{| class="wikitable"
|+ Table mapping token values to BASIC keywords
! title="The value of the token in hexadecimal" | Hex
! title="The value of the token in decimal" | Dec
! title="The BASIC keyword the token stands for" | Keyword
! title="Miscellany" | Notes
|-
|-
|80||128||END
|-
|81||129||FOR
|-
|82||130||NEXT
|-
|83||131||DATA
|-
|84||132||INPUT
|-
|85||133||DIM
|-
|86||134||READ
|-
|87||135||LET
|-
|88||136||GOTO
|-
|89||137||RUN
|-
|8A||138||IF
|-
|8B||139||RESTORE
|-
|8C||140||GOSUB
|-
|8D||141||RETURN
|-
|8E||142||REM
|-
|8F||143||STOP
|-
|90||144||WIDTH
|-
|91||145||ELSE||When tokenizing, the Tandy 200 always adds a colon (':') before the ELSE token (3A 91). If the user actually writes :ELSE it is tokenized as 3A 3A 91.
|-
|92||146||LINE
|-
|93||147||EDIT
|-
|94||148||ERROR
|-
|95||149||RESUME
|-
|96||150||OUT
|-
|97||151||ON
|-
|98||152||DSKO$
|-
|99||153||OPEN
|-
|9A||154||CLOSE
|-
|9B||155||LOAD||Note that LOADM is simply the token for LOAD followed by an ASCII 'M'.
|-
|9C||156||MERGE
|-QUOTE
|9D||157||FILES
|-
|9E||158||SAVE
|-
|9F||159||LFILES
|-
|A0||160||LPRINT
|-
|A1||161||DEF
|-
|A2||162||POKE
|-
|A3||163||PRINT
|-
|A4||164||CONT
|-
|A5||165||LIST
|-
|A6||166||LLIST
|-
|A7||167||CLEAR
|-
|A8||168||CLOAD
|-
|A9||169||CSAVE
|-
|AA||170||TIME$
|-
|AB||171||DATE$
|-
|AC||172||DAY$
|-
|AD||173||COM
|-
|AE||174||MDM
|-
|AF||175||KEY
|-
|B0||176||CLS
|-
|B1||177||BEEP
|-
|B2||178||SOUND
|-
|B3||179||LCOPY
|-
|B4||180||PSET
|-
|B5||181||PRESET
|-
|B6||182||MOTOR
|-
|B7||183||MAX
|-
|B8||184||POWER
|-
|B9||185||CALL
|-
|BA||186||MENU
|-
|BB||187||IPL
|-
|BC||188||NAME
|-
|BD||189||KILL
|-
|BE||190||SCREEN
|-
|BF||191||NEW
|-
|C0||192||TAB(
|-
|C1||193||TO
|-
|C2||194||USING
|-
|C3||195||VARPTR
|-
|C4||196||ERL
|-
|C5||197||ERR
|-
|C6||198||STRING$
|-
|C7||199||INSTR
|-
|C8||200||DSKI$
|-
|C9||201||INKEY$
|-
|CA||202||CSRLIN
|-
|CB||203||OFF
|-
|CC||204||HIMEM
|-
|CD||205||THEN
|-
|CE||206||NOT
|-
|CF||207||STEP
|-
|D0||208||+
|-
|D1||209||-
|-
|D2||210||*
|-
|D3||211||/
|-
|D4||212||^
|-
|D5||213||AND
|-
|D6||214||OR
|-
|D7||215||XOR
|-
|D8||216||EQV
|-
|D9||217||IMP
|-
|DA||218||MOD
|-
|DB||219||\
|-
|DC||220||>
|-
|DD||221||=
|-
|DE||222||<
|-
|DF||223||SGN
|-
|E0||224||INT
|-
|E1||225||ABS
|-
|E2||226||FRE
|-
|E3||227||INP
|-
|E4||228||LPOS
|-
|E5||229||POS
|-
|E6||230||SQR
|-
|E7||231||RND
|-
|E8||232||LOG
|-
|E9||233||EXP
|-
|EA||234||COS
|-
|EB||235||SIN
|-
|EC||236||TAN
|-
|ED||237||ATN
|-
|EE||238||PEEK
|-
|EF||239||EOF
|-
|F0||240||LOC
|-
|F1||241||LOF
|-
|F2||242||CINT
|-
|F3||243||CSNG
|-
|F4||244||CDBL
|-
|F5||245||FIX
|-
|F6||246||LEN
|-
|F7||247||STR$
|-
|F8||248||VAL
|-
|F9||249||ASC
|-
|FA||250||CHR$
|-
|FB||251||SPACE$
|-
|FC||252||LEFT$
|-
|FD||253||RIGHT$
|-
|FE||254||MID$
|-
|FF||255||' (QUOTE) || When tokenizing, the single quote character expands to three characters: a colon (3A), the byte for REM (8E), and then FF.
|}

== File Format ==

Tokenised BASIC code is a sequence of tokenized lines. Each tokenized line has the following format:

{|class="wikitable"
|+ Format of a single tokenized line
!Name!!Length!!Description
|-
|PL PH
|2 bytes
|Address in RAM of the next line of BASIC; unsigned 16-bit integer, little-endian. PL PH equal to zero marks the end of a tokenized BASIC listing. All other values can be ignored as they are merely placeholders in the file. (See [[#Details for PL PH]].)
|-
|LL LH
|2 bytes
|Line number; unsigned 16-bit integer, little-endian.
|-
|B0 … Bn
|Any number of bytes
|Depending on the high bit, each byte is either a token or an ASCII character.
|-
|NULL
|1 byte
|NULL byte (<code>0x00</code>) to signal end of line.
|}

The end of the tokenized data is signaled by two consecutive NULLs in the location where PL PH would normally begin a new line.

=== Limits ===

* Line numbers can range from 0 to 65529 (but see variances below).
* Tokenized line length is limited to 256 characters, including the trailing NULL byte. (To do: double check this.)

=== Acceptable variances ===

While Tandy BASIC will never generate such files, it has no problem loading a tokenized program that:

* Has line numbers out of order; they will be sorted when loaded.
* Contains duplicate line numbers; previous line is discarded.
* Contains arbitrary bytes for PL PH; they are always regenerated during loading.
* Has a ^Z (ASCII 26) at the end of the file.
* Contains line numbers greater than the nominal maximum of 65529; this can be used to store hidden binary data [https://www.mail-archive.com/m100@lists.bitchin100.com/msg16195.html according to John R. Hogerhuis].
* Contains a tokenized line with zero bytes of data; if a line with that number existed before, it will be discarded.

''Sidenote'': while these variances are accepted by actual hardware, emulators may not be able to handle such files. For example, [https://sourceforge.net/projects/virtualt/ Virtual T] (as of version 1.7 in 2022) refuses to load tokenized BASIC programs that have lines out of order, duplicate lines, or bogus values for PL PH.

=== Details for PL PH ===

The value of the 16-bit integer PL PH in a tokenized BASIC file is not used, except to mark the end of the listing when PL PH equals zero. During program load, Tandy BASIC always recalculates PL PH. Once in memory, PL PH is used to skip to the line required for GOTO, GOSUB, and READ, thus speeding up the program.

(Stub. This is where the correct calculation of PL PH values will go.)

== Format documentation ==

* [https://m100.bitchin100.narkive.com/eP2uSl4J/tokenized-basic-programs-bytes-of-mystery Discussion of byte format on Bitchin 100 Mailing List]
* [http://www.club100.org/library/ups/tokens.do BASIC program to create a table of tokens]
* [https://archive.org/details/HiddenPowersOfTheTrs80Model100/page/n229/mode/2up?view=theater Hidden Powers of the TRS 80 Model 100]. Appendix B lists the BASIC token values for the Model 100. Other than the omission of 255 for a single quote remark, this reference matches what hackerb9 found programmatically for the Tandy 200.
* [http://web.archive.org/web/20220324135218/https://help.ayra.ch/trs80-reference#trs-80-model-100-basic-reference Ayra Model 100 BASIC Reference]

== Other links ==

* [https://github.com/hackerb9/tokenize/ Hackerb9's simple program for tokenizing BASIC code]
* [https://sourceforge.net/projects/virtualt/ A Tandy 200 & Model 100/102 emulator]
* [http://bitchin100.com/CloudT/ A TRS-80 Model 100 emulator that runs in a web browser]

[[Category:Microsoft]]

Tandy 200 BASIC tokenized file

2022-10-05T21:37:20Z

Hackerb9: /* Acceptable variances */ Be specific about "illegal line numbers"

{{FormatInfo
|subcat=Source code
|subcat2=Tokenized BASIC
|released=1984
}}

'''Tandy 200 BASIC''' was a version of Microsoft BASIC for the Radio Shack Tandy 200 computer. The tokenizations for the TRS-80 Model 100 and Tandy 102 appear to be identical. The NEC PC-8201/8300 format is not the same, but should be similar.

== Tokens ==
All BASIC tokens on the Tandy 200 are a single byte with a value from 128 to 255.

{| class="wikitable"
|+ Table mapping token values to BASIC keywords
! title="The value of the token in hexadecimal" | Hex
! title="The value of the token in decimal" | Dec
! title="The BASIC keyword the token stands for" | Keyword
! title="Miscellany" | Notes
|-
|-
|80||128||END
|-
|81||129||FOR
|-
|82||130||NEXT
|-
|83||131||DATA
|-
|84||132||INPUT
|-
|85||133||DIM
|-
|86||134||READ
|-
|87||135||LET
|-
|88||136||GOTO
|-
|89||137||RUN
|-
|8A||138||IF
|-
|8B||139||RESTORE
|-
|8C||140||GOSUB
|-
|8D||141||RETURN
|-
|8E||142||REM
|-
|8F||143||STOP
|-
|90||144||WIDTH
|-
|91||145||ELSE||When tokenizing, the Tandy 200 always adds a colon (':') before the ELSE token (3A 91). If the user actually writes :ELSE it is tokenized as 3A 3A 91.
|-
|92||146||LINE
|-
|93||147||EDIT
|-
|94||148||ERROR
|-
|95||149||RESUME
|-
|96||150||OUT
|-
|97||151||ON
|-
|98||152||DSKO$
|-
|99||153||OPEN
|-
|9A||154||CLOSE
|-
|9B||155||LOAD||Note that LOADM is simply the token for LOAD followed by an ASCII 'M'.
|-
|9C||156||MERGE
|-QUOTE
|9D||157||FILES
|-
|9E||158||SAVE
|-
|9F||159||LFILES
|-
|A0||160||LPRINT
|-
|A1||161||DEF
|-
|A2||162||POKE
|-
|A3||163||PRINT
|-
|A4||164||CONT
|-
|A5||165||LIST
|-
|A6||166||LLIST
|-
|A7||167||CLEAR
|-
|A8||168||CLOAD
|-
|A9||169||CSAVE
|-
|AA||170||TIME$
|-
|AB||171||DATE$
|-
|AC||172||DAY$
|-
|AD||173||COM
|-
|AE||174||MDM
|-
|AF||175||KEY
|-
|B0||176||CLS
|-
|B1||177||BEEP
|-
|B2||178||SOUND
|-
|B3||179||LCOPY
|-
|B4||180||PSET
|-
|B5||181||PRESET
|-
|B6||182||MOTOR
|-
|B7||183||MAX
|-
|B8||184||POWER
|-
|B9||185||CALL
|-
|BA||186||MENU
|-
|BB||187||IPL
|-
|BC||188||NAME
|-
|BD||189||KILL
|-
|BE||190||SCREEN
|-
|BF||191||NEW
|-
|C0||192||TAB(
|-
|C1||193||TO
|-
|C2||194||USING
|-
|C3||195||VARPTR
|-
|C4||196||ERL
|-
|C5||197||ERR
|-
|C6||198||STRING$
|-
|C7||199||INSTR
|-
|C8||200||DSKI$
|-
|C9||201||INKEY$
|-
|CA||202||CSRLIN
|-
|CB||203||OFF
|-
|CC||204||HIMEM
|-
|CD||205||THEN
|-
|CE||206||NOT
|-
|CF||207||STEP
|-
|D0||208||+
|-
|D1||209||-
|-
|D2||210||*
|-
|D3||211||/
|-
|D4||212||^
|-
|D5||213||AND
|-
|D6||214||OR
|-
|D7||215||XOR
|-
|D8||216||EQV
|-
|D9||217||IMP
|-
|DA||218||MOD
|-
|DB||219||\
|-
|DC||220||>
|-
|DD||221||=
|-
|DE||222||<
|-
|DF||223||SGN
|-
|E0||224||INT
|-
|E1||225||ABS
|-
|E2||226||FRE
|-
|E3||227||INP
|-
|E4||228||LPOS
|-
|E5||229||POS
|-
|E6||230||SQR
|-
|E7||231||RND
|-
|E8||232||LOG
|-
|E9||233||EXP
|-
|EA||234||COS
|-
|EB||235||SIN
|-
|EC||236||TAN
|-
|ED||237||ATN
|-
|EE||238||PEEK
|-
|EF||239||EOF
|-
|F0||240||LOC
|-
|F1||241||LOF
|-
|F2||242||CINT
|-
|F3||243||CSNG
|-
|F4||244||CDBL
|-
|F5||245||FIX
|-
|F6||246||LEN
|-
|F7||247||STR$
|-
|F8||248||VAL
|-
|F9||249||ASC
|-
|FA||250||CHR$
|-
|FB||251||SPACE$
|-
|FC||252||LEFT$
|-
|FD||253||RIGHT$
|-
|FE||254||MID$
|-
|FF||255||' (QUOTE) || When tokenizing, the single quote character expands to three characters: a colon (3A), the byte for REM (8E), and then FF.
|}

== File Format ==

Tokenised BASIC code is a sequence of tokenized lines. Each tokenized line has the following format:

{|class="wikitable"
|+ Format of a single tokenized line
!Name!!Length!!Description
|-
|PL PH
|2 bytes
|Address in RAM of the next line of BASIC; unsigned 16-bit integer, little-endian. PL PH equal to zero marks the end of a tokenized BASIC listing. All other values can be ignored as they are merely placeholders in the file. (See [[#Details for PL PH]].)
|-
|LL LH
|2 bytes
|Line number; unsigned 16-bit integer, little-endian.
|-
|B0 … Bn
|Any number of bytes
|Depending on the high bit, each byte is either a token or an ASCII character.
|-
|NULL
|1 byte
|NULL byte (<code>0x00</code>) to signal end of line.
|}

The end of the tokenized data is signaled by two consecutive NULLs in the location where PL PH would normally begin a new line.

=== Limits ===

* Line numbers can range from 0 to 65529 (but see variances below).
* Tokenized line length is limited to 256 characters, including the trailing NULL byte. (To do: double check this.)

=== Acceptable variances ===

While Tandy BASIC will never generate such files, it has no problem loading a tokenized program that:

* Has line numbers out of order; they will be sorted when loaded.
* Contains duplicate line numbers; previous line is discarded.
* Contains arbitrary bytes for PL PH; they are always regenerated during loading.
* Has a ^Z (ASCII 26) at the end of the file.
* Contains line numbers greater than the nominal maximum of 65529; this can be used to store hidden binary data [https://www.mail-archive.com/m100@lists.bitchin100.com/msg16195.html according to John R. Hogerhuis].

Note that while these variances are accepted by actual hardware, emulators may not be able to handle such files. For example, the [https://sourceforge.net/projects/virtualt/ Virtual T] emulator (as of version 1.7 in 2022) refuses to load tokenized BASIC programs that have lines out of order, duplicate lines, or bogus values for PL PH.

=== Details for PL PH ===

The value of the 16-bit integer PL PH in a tokenized BASIC file is not used, except to mark the end of the listing when PL PH equals zero. During program load, Tandy BASIC always recalculates PL PH. Once in memory, PL PH is used to skip to the line required for GOTO, GOSUB, and READ, thus speeding up the program.

(Stub. This is where the correct calculation of PL PH values will go.)

== Format documentation ==

* [https://m100.bitchin100.narkive.com/eP2uSl4J/tokenized-basic-programs-bytes-of-mystery Discussion of byte format on Bitchin 100 Mailing List]
* [http://www.club100.org/library/ups/tokens.do BASIC program to create a table of tokens]
* [https://archive.org/details/HiddenPowersOfTheTrs80Model100/page/n229/mode/2up?view=theater Hidden Powers of the TRS 80 Model 100]. Appendix B lists the BASIC token values for the Model 100. Other than the omission of 255 for a single quote remark, this reference matches what hackerb9 found programmatically for the Tandy 200.
* [http://web.archive.org/web/20220324135218/https://help.ayra.ch/trs80-reference#trs-80-model-100-basic-reference Ayra Model 100 BASIC Reference]

== Other links ==

* [https://github.com/hackerb9/tokenize/ Hackerb9's simple program for tokenizing BASIC code]
* [https://sourceforge.net/projects/virtualt/ A Tandy 200 & Model 100/102 emulator]
* [http://bitchin100.com/CloudT/ A TRS-80 Model 100 emulator that runs in a web browser]

[[Category:Microsoft]]

Tandy 200 BASIC tokenized file

2022-10-05T21:32:16Z

Hackerb9: Move PL PH details to section instead of putting inside table

{{FormatInfo
|subcat=Source code
|subcat2=Tokenized BASIC
|released=1984
}}

'''Tandy 200 BASIC''' was a version of Microsoft BASIC for the Radio Shack Tandy 200 computer. The tokenizations for the TRS-80 Model 100 and Tandy 102 appear to be identical. The NEC PC-8201/8300 format is not the same, but should be similar.

== Tokens ==
All BASIC tokens on the Tandy 200 are a single byte with a value from 128 to 255.

{| class="wikitable"
|+ Table mapping token values to BASIC keywords
! title="The value of the token in hexadecimal" | Hex
! title="The value of the token in decimal" | Dec
! title="The BASIC keyword the token stands for" | Keyword
! title="Miscellany" | Notes
|-
|-
|80||128||END
|-
|81||129||FOR
|-
|82||130||NEXT
|-
|83||131||DATA
|-
|84||132||INPUT
|-
|85||133||DIM
|-
|86||134||READ
|-
|87||135||LET
|-
|88||136||GOTO
|-
|89||137||RUN
|-
|8A||138||IF
|-
|8B||139||RESTORE
|-
|8C||140||GOSUB
|-
|8D||141||RETURN
|-
|8E||142||REM
|-
|8F||143||STOP
|-
|90||144||WIDTH
|-
|91||145||ELSE||When tokenizing, the Tandy 200 always adds a colon (':') before the ELSE token (3A 91). If the user actually writes :ELSE it is tokenized as 3A 3A 91.
|-
|92||146||LINE
|-
|93||147||EDIT
|-
|94||148||ERROR
|-
|95||149||RESUME
|-
|96||150||OUT
|-
|97||151||ON
|-
|98||152||DSKO$
|-
|99||153||OPEN
|-
|9A||154||CLOSE
|-
|9B||155||LOAD||Note that LOADM is simply the token for LOAD followed by an ASCII 'M'.
|-
|9C||156||MERGE
|-QUOTE
|9D||157||FILES
|-
|9E||158||SAVE
|-
|9F||159||LFILES
|-
|A0||160||LPRINT
|-
|A1||161||DEF
|-
|A2||162||POKE
|-
|A3||163||PRINT
|-
|A4||164||CONT
|-
|A5||165||LIST
|-
|A6||166||LLIST
|-
|A7||167||CLEAR
|-
|A8||168||CLOAD
|-
|A9||169||CSAVE
|-
|AA||170||TIME$
|-
|AB||171||DATE$
|-
|AC||172||DAY$
|-
|AD||173||COM
|-
|AE||174||MDM
|-
|AF||175||KEY
|-
|B0||176||CLS
|-
|B1||177||BEEP
|-
|B2||178||SOUND
|-
|B3||179||LCOPY
|-
|B4||180||PSET
|-
|B5||181||PRESET
|-
|B6||182||MOTOR
|-
|B7||183||MAX
|-
|B8||184||POWER
|-
|B9||185||CALL
|-
|BA||186||MENU
|-
|BB||187||IPL
|-
|BC||188||NAME
|-
|BD||189||KILL
|-
|BE||190||SCREEN
|-
|BF||191||NEW
|-
|C0||192||TAB(
|-
|C1||193||TO
|-
|C2||194||USING
|-
|C3||195||VARPTR
|-
|C4||196||ERL
|-
|C5||197||ERR
|-
|C6||198||STRING$
|-
|C7||199||INSTR
|-
|C8||200||DSKI$
|-
|C9||201||INKEY$
|-
|CA||202||CSRLIN
|-
|CB||203||OFF
|-
|CC||204||HIMEM
|-
|CD||205||THEN
|-
|CE||206||NOT
|-
|CF||207||STEP
|-
|D0||208||+
|-
|D1||209||-
|-
|D2||210||*
|-
|D3||211||/
|-
|D4||212||^
|-
|D5||213||AND
|-
|D6||214||OR
|-
|D7||215||XOR
|-
|D8||216||EQV
|-
|D9||217||IMP
|-
|DA||218||MOD
|-
|DB||219||\
|-
|DC||220||>
|-
|DD||221||=
|-
|DE||222||<
|-
|DF||223||SGN
|-
|E0||224||INT
|-
|E1||225||ABS
|-
|E2||226||FRE
|-
|E3||227||INP
|-
|E4||228||LPOS
|-
|E5||229||POS
|-
|E6||230||SQR
|-
|E7||231||RND
|-
|E8||232||LOG
|-
|E9||233||EXP
|-
|EA||234||COS
|-
|EB||235||SIN
|-
|EC||236||TAN
|-
|ED||237||ATN
|-
|EE||238||PEEK
|-
|EF||239||EOF
|-
|F0||240||LOC
|-
|F1||241||LOF
|-
|F2||242||CINT
|-
|F3||243||CSNG
|-
|F4||244||CDBL
|-
|F5||245||FIX
|-
|F6||246||LEN
|-
|F7||247||STR$
|-
|F8||248||VAL
|-
|F9||249||ASC
|-
|FA||250||CHR$
|-
|FB||251||SPACE$
|-
|FC||252||LEFT$
|-
|FD||253||RIGHT$
|-
|FE||254||MID$
|-
|FF||255||' (QUOTE) || When tokenizing, the single quote character expands to three characters: a colon (3A), the byte for REM (8E), and then FF.
|}

== File Format ==

Tokenised BASIC code is a sequence of tokenized lines. Each tokenized line has the following format:

{|class="wikitable"
|+ Format of a single tokenized line
!Name!!Length!!Description
|-
|PL PH
|2 bytes
|Address in RAM of the next line of BASIC; unsigned 16-bit integer, little-endian. PL PH equal to zero marks the end of a tokenized BASIC listing. All other values can be ignored as they are merely placeholders in the file. (See [[#Details for PL PH]].)
|-
|LL LH
|2 bytes
|Line number; unsigned 16-bit integer, little-endian.
|-
|B0 … Bn
|Any number of bytes
|Depending on the high bit, each byte is either a token or an ASCII character.
|-
|NULL
|1 byte
|NULL byte (<code>0x00</code>) to signal end of line.
|}

The end of the tokenized data is signaled by two consecutive NULLs in the location where PL PH would normally begin a new line.

=== Limits ===

* Line numbers can range from 0 to 65529 (but see variances below).
* Tokenized line length is limited to 256 characters, including the trailing NULL byte. (To do: double check this.)

=== Acceptable variances ===

While Tandy BASIC will never generate such files, it has no problem loading a tokenized program that:

* Has line numbers out of order; they will be sorted when loaded.
* Contains duplicate line numbers; previous line is discarded.
* Contains arbitrary bytes for PL PH; they are always regenerated during loading.
* Has a ^Z (ASCII 26) at the end of the file.
* Contains illegal line numbers; this can be used to store hidden binary data [https://www.mail-archive.com/m100@lists.bitchin100.com/msg16195.html according to John R. Hogerhuis].

Note that while these variances are accepted on an actual Model T device (Model 100, Tandy 200, or Tandy 102), emulators may not be able to handle such files. For example, the [https://sourceforge.net/projects/virtualt/ Virtual T] emulator (as of version 1.7 in 2022) refuses to load tokenized BASIC programs that have lines out of order, duplicate lines, or bogus values for PL PH.

=== Details for PL PH ===

The value of the 16-bit integer PL PH in a tokenized BASIC file is not used, except to mark the end of the listing when PL PH equals zero. During program load, Tandy BASIC always recalculates PL PH. Once in memory, PL PH is used to skip to the line required for GOTO, GOSUB, and READ, thus speeding up the program.

(Stub. This is where the correct calculation of PL PH values will go.)

== Format documentation ==

* [https://m100.bitchin100.narkive.com/eP2uSl4J/tokenized-basic-programs-bytes-of-mystery Discussion of byte format on Bitchin 100 Mailing List]
* [http://www.club100.org/library/ups/tokens.do BASIC program to create a table of tokens]
* [https://archive.org/details/HiddenPowersOfTheTrs80Model100/page/n229/mode/2up?view=theater Hidden Powers of the TRS 80 Model 100]. Appendix B lists the BASIC token values for the Model 100. Other than the omission of 255 for a single quote remark, this reference matches what hackerb9 found programmatically for the Tandy 200.
* [http://web.archive.org/web/20220324135218/https://help.ayra.ch/trs80-reference#trs-80-model-100-basic-reference Ayra Model 100 BASIC Reference]

== Other links ==

* [https://github.com/hackerb9/tokenize/ Hackerb9's simple program for tokenizing BASIC code]
* [https://sourceforge.net/projects/virtualt/ A Tandy 200 & Model 100/102 emulator]
* [http://bitchin100.com/CloudT/ A TRS-80 Model 100 emulator that runs in a web browser]

[[Category:Microsoft]]

Tandy 200 BASIC tokenized file

2022-10-05T21:22:34Z

Hackerb9: /* File Format */ Shorten description by simply saying "high bit" instead of talking about >= 128.

{{FormatInfo
|subcat=Source code
|subcat2=Tokenized BASIC
|released=1984
}}

'''Tandy 200 BASIC''' was a version of Microsoft BASIC for the Radio Shack Tandy 200 computer. The tokenizations for the TRS-80 Model 100 and Tandy 102 appear to be identical. The NEC PC-8201/8300 format is not the same, but should be similar.

== Tokens ==
All BASIC tokens on the Tandy 200 are a single byte with a value from 128 to 255.

{| class="wikitable"
|+ Table mapping token values to BASIC keywords
! title="The value of the token in hexadecimal" | Hex
! title="The value of the token in decimal" | Dec
! title="The BASIC keyword the token stands for" | Keyword
! title="Miscellany" | Notes
|-
|-
|80||128||END
|-
|81||129||FOR
|-
|82||130||NEXT
|-
|83||131||DATA
|-
|84||132||INPUT
|-
|85||133||DIM
|-
|86||134||READ
|-
|87||135||LET
|-
|88||136||GOTO
|-
|89||137||RUN
|-
|8A||138||IF
|-
|8B||139||RESTORE
|-
|8C||140||GOSUB
|-
|8D||141||RETURN
|-
|8E||142||REM
|-
|8F||143||STOP
|-
|90||144||WIDTH
|-
|91||145||ELSE||When tokenizing, the Tandy 200 always adds a colon (':') before the ELSE token (3A 91). If the user actually writes :ELSE it is tokenized as 3A 3A 91.
|-
|92||146||LINE
|-
|93||147||EDIT
|-
|94||148||ERROR
|-
|95||149||RESUME
|-
|96||150||OUT
|-
|97||151||ON
|-
|98||152||DSKO$
|-
|99||153||OPEN
|-
|9A||154||CLOSE
|-
|9B||155||LOAD||Note that LOADM is simply the token for LOAD followed by an ASCII 'M'.
|-
|9C||156||MERGE
|-QUOTE
|9D||157||FILES
|-
|9E||158||SAVE
|-
|9F||159||LFILES
|-
|A0||160||LPRINT
|-
|A1||161||DEF
|-
|A2||162||POKE
|-
|A3||163||PRINT
|-
|A4||164||CONT
|-
|A5||165||LIST
|-
|A6||166||LLIST
|-
|A7||167||CLEAR
|-
|A8||168||CLOAD
|-
|A9||169||CSAVE
|-
|AA||170||TIME$
|-
|AB||171||DATE$
|-
|AC||172||DAY$
|-
|AD||173||COM
|-
|AE||174||MDM
|-
|AF||175||KEY
|-
|B0||176||CLS
|-
|B1||177||BEEP
|-
|B2||178||SOUND
|-
|B3||179||LCOPY
|-
|B4||180||PSET
|-
|B5||181||PRESET
|-
|B6||182||MOTOR
|-
|B7||183||MAX
|-
|B8||184||POWER
|-
|B9||185||CALL
|-
|BA||186||MENU
|-
|BB||187||IPL
|-
|BC||188||NAME
|-
|BD||189||KILL
|-
|BE||190||SCREEN
|-
|BF||191||NEW
|-
|C0||192||TAB(
|-
|C1||193||TO
|-
|C2||194||USING
|-
|C3||195||VARPTR
|-
|C4||196||ERL
|-
|C5||197||ERR
|-
|C6||198||STRING$
|-
|C7||199||INSTR
|-
|C8||200||DSKI$
|-
|C9||201||INKEY$
|-
|CA||202||CSRLIN
|-
|CB||203||OFF
|-
|CC||204||HIMEM
|-
|CD||205||THEN
|-
|CE||206||NOT
|-
|CF||207||STEP
|-
|D0||208||+
|-
|D1||209||-
|-
|D2||210||*
|-
|D3||211||/
|-
|D4||212||^
|-
|D5||213||AND
|-
|D6||214||OR
|-
|D7||215||XOR
|-
|D8||216||EQV
|-
|D9||217||IMP
|-
|DA||218||MOD
|-
|DB||219||\
|-
|DC||220||>
|-
|DD||221||=
|-
|DE||222||<
|-
|DF||223||SGN
|-
|E0||224||INT
|-
|E1||225||ABS
|-
|E2||226||FRE
|-
|E3||227||INP
|-
|E4||228||LPOS
|-
|E5||229||POS
|-
|E6||230||SQR
|-
|E7||231||RND
|-
|E8||232||LOG
|-
|E9||233||EXP
|-
|EA||234||COS
|-
|EB||235||SIN
|-
|EC||236||TAN
|-
|ED||237||ATN
|-
|EE||238||PEEK
|-
|EF||239||EOF
|-
|F0||240||LOC
|-
|F1||241||LOF
|-
|F2||242||CINT
|-
|F3||243||CSNG
|-
|F4||244||CDBL
|-
|F5||245||FIX
|-
|F6||246||LEN
|-
|F7||247||STR$
|-
|F8||248||VAL
|-
|F9||249||ASC
|-
|FA||250||CHR$
|-
|FB||251||SPACE$
|-
|FC||252||LEFT$
|-
|FD||253||RIGHT$
|-
|FE||254||MID$
|-
|FF||255||' (QUOTE) || When tokenizing, the single quote character expands to three characters: a colon (3A), the byte for REM (8E), and then FF.
|}

== File Format ==

Tokenised BASIC code is a sequence of tokenized lines. Each tokenized line has the following format:

{|class="wikitable"
|+ Format of a single tokenized line
!Name!!Length!!Description
|-
|PL PH
|2 bytes
|Address in RAM of the next line of BASIC; unsigned 16-bit integer, little-endian. PL PH equal to zero marks the end of a tokenized BASIC listing. All other values can be ignored as they are merely placeholders in the file. During program load, Tandy BASIC always recalculates PL PH. Once in memory, PL PH is used to skip to the line required for GOTO, GOSUB, and READ, thus speeding up the program.
|-
|LL LH
|2 bytes
|Line number; unsigned 16-bit integer, little-endian.
|-
|B0 … Bn
|Any number of bytes
|Depending on the high bit, each byte is either a token or an ASCII character.
|-
|NULL
|1 byte
|NULL byte (<code>0x00</code>) to signal end of line.
|}

The end of the tokenized data is signaled by two consecutive NULLs in the location where PL PH would normally begin a new line.

=== Limits ===

* Line numbers can range from 0 to 65529 (but see variances below).
* Tokenized line length is limited to 256 characters, including the trailing NULL byte. (To do: double check this.)

=== Acceptable variances ===

While Tandy BASIC will never generate such files, it has no problem loading a tokenized program that:

* Has line numbers out of order; they will be sorted when loaded.
* Contains duplicate line numbers; previous line is discarded.
* Contains arbitrary bytes for PL PH; they are always regenerated during loading.
* Has a ^Z (ASCII 26) at the end of the file.
* Contains illegal line numbers; this can be used to store hidden binary data [https://www.mail-archive.com/m100@lists.bitchin100.com/msg16195.html according to John R. Hogerhuis].

Note that while these variances are accepted on an actual Model T device (Model 100, Tandy 200, or Tandy 102), emulators may not be able to handle such files. For example, the [https://sourceforge.net/projects/virtualt/ Virtual T] emulator (as of version 1.7 in 2022) refuses to load tokenized BASIC programs that have lines out of order, duplicate lines, or bogus values for PL PH.

=== Details for PL PH ===

(Stub. This is where the correct calculation of PL PH values will go.)

== Format documentation ==

* [https://m100.bitchin100.narkive.com/eP2uSl4J/tokenized-basic-programs-bytes-of-mystery Discussion of byte format on Bitchin 100 Mailing List]
* [http://www.club100.org/library/ups/tokens.do BASIC program to create a table of tokens]
* [https://archive.org/details/HiddenPowersOfTheTrs80Model100/page/n229/mode/2up?view=theater Hidden Powers of the TRS 80 Model 100]. Appendix B lists the BASIC token values for the Model 100. Other than the omission of 255 for a single quote remark, this reference matches what hackerb9 found programmatically for the Tandy 200.
* [http://web.archive.org/web/20220324135218/https://help.ayra.ch/trs80-reference#trs-80-model-100-basic-reference Ayra Model 100 BASIC Reference]

== Other links ==

* [https://github.com/hackerb9/tokenize/ Hackerb9's simple program for tokenizing BASIC code]
* [https://sourceforge.net/projects/virtualt/ A Tandy 200 & Model 100/102 emulator]
* [http://bitchin100.com/CloudT/ A TRS-80 Model 100 emulator that runs in a web browser]

[[Category:Microsoft]]

Tandy 200 BASIC tokenized file

2022-10-05T21:09:24Z

Hackerb9: /* Tokens */ Add caption for table of tokens

{{FormatInfo
|subcat=Source code
|subcat2=Tokenized BASIC
|released=1984
}}

'''Tandy 200 BASIC''' was a version of Microsoft BASIC for the Radio Shack Tandy 200 computer. The tokenizations for the TRS-80 Model 100 and Tandy 102 appear to be identical. The NEC PC-8201/8300 format is not the same, but should be similar.

== Tokens ==
All BASIC tokens on the Tandy 200 are a single byte with a value from 128 to 255.

{| class="wikitable"
|+ Table mapping token values to BASIC keywords
! title="The value of the token in hexadecimal" | Hex
! title="The value of the token in decimal" | Dec
! title="The BASIC keyword the token stands for" | Keyword
! title="Miscellany" | Notes
|-
|-
|80||128||END
|-
|81||129||FOR
|-
|82||130||NEXT
|-
|83||131||DATA
|-
|84||132||INPUT
|-
|85||133||DIM
|-
|86||134||READ
|-
|87||135||LET
|-
|88||136||GOTO
|-
|89||137||RUN
|-
|8A||138||IF
|-
|8B||139||RESTORE
|-
|8C||140||GOSUB
|-
|8D||141||RETURN
|-
|8E||142||REM
|-
|8F||143||STOP
|-
|90||144||WIDTH
|-
|91||145||ELSE||When tokenizing, the Tandy 200 always adds a colon (':') before the ELSE token (3A 91). If the user actually writes :ELSE it is tokenized as 3A 3A 91.
|-
|92||146||LINE
|-
|93||147||EDIT
|-
|94||148||ERROR
|-
|95||149||RESUME
|-
|96||150||OUT
|-
|97||151||ON
|-
|98||152||DSKO$
|-
|99||153||OPEN
|-
|9A||154||CLOSE
|-
|9B||155||LOAD||Note that LOADM is simply the token for LOAD followed by an ASCII 'M'.
|-
|9C||156||MERGE
|-QUOTE
|9D||157||FILES
|-
|9E||158||SAVE
|-
|9F||159||LFILES
|-
|A0||160||LPRINT
|-
|A1||161||DEF
|-
|A2||162||POKE
|-
|A3||163||PRINT
|-
|A4||164||CONT
|-
|A5||165||LIST
|-
|A6||166||LLIST
|-
|A7||167||CLEAR
|-
|A8||168||CLOAD
|-
|A9||169||CSAVE
|-
|AA||170||TIME$
|-
|AB||171||DATE$
|-
|AC||172||DAY$
|-
|AD||173||COM
|-
|AE||174||MDM
|-
|AF||175||KEY
|-
|B0||176||CLS
|-
|B1||177||BEEP
|-
|B2||178||SOUND
|-
|B3||179||LCOPY
|-
|B4||180||PSET
|-
|B5||181||PRESET
|-
|B6||182||MOTOR
|-
|B7||183||MAX
|-
|B8||184||POWER
|-
|B9||185||CALL
|-
|BA||186||MENU
|-
|BB||187||IPL
|-
|BC||188||NAME
|-
|BD||189||KILL
|-
|BE||190||SCREEN
|-
|BF||191||NEW
|-
|C0||192||TAB(
|-
|C1||193||TO
|-
|C2||194||USING
|-
|C3||195||VARPTR
|-
|C4||196||ERL
|-
|C5||197||ERR
|-
|C6||198||STRING$
|-
|C7||199||INSTR
|-
|C8||200||DSKI$
|-
|C9||201||INKEY$
|-
|CA||202||CSRLIN
|-
|CB||203||OFF
|-
|CC||204||HIMEM
|-
|CD||205||THEN
|-
|CE||206||NOT
|-
|CF||207||STEP
|-
|D0||208||+
|-
|D1||209||-
|-
|D2||210||*
|-
|D3||211||/
|-
|D4||212||^
|-
|D5||213||AND
|-
|D6||214||OR
|-
|D7||215||XOR
|-
|D8||216||EQV
|-
|D9||217||IMP
|-
|DA||218||MOD
|-
|DB||219||\
|-
|DC||220||>
|-
|DD||221||=
|-
|DE||222||<
|-
|DF||223||SGN
|-
|E0||224||INT
|-
|E1||225||ABS
|-
|E2||226||FRE
|-
|E3||227||INP
|-
|E4||228||LPOS
|-
|E5||229||POS
|-
|E6||230||SQR
|-
|E7||231||RND
|-
|E8||232||LOG
|-
|E9||233||EXP
|-
|EA||234||COS
|-
|EB||235||SIN
|-
|EC||236||TAN
|-
|ED||237||ATN
|-
|EE||238||PEEK
|-
|EF||239||EOF
|-
|F0||240||LOC
|-
|F1||241||LOF
|-
|F2||242||CINT
|-
|F3||243||CSNG
|-
|F4||244||CDBL
|-
|F5||245||FIX
|-
|F6||246||LEN
|-
|F7||247||STR$
|-
|F8||248||VAL
|-
|F9||249||ASC
|-
|FA||250||CHR$
|-
|FB||251||SPACE$
|-
|FC||252||LEFT$
|-
|FD||253||RIGHT$
|-
|FE||254||MID$
|-
|FF||255||' (QUOTE) || When tokenizing, the single quote character expands to three characters: a colon (3A), the byte for REM (8E), and then FF.
|}

== File Format ==

Tokenised BASIC code is a sequence of tokenized lines. Each tokenized line has the following format:

{|class="wikitable"
|+ Format of a single tokenized line
!Name!!Length!!Description
|-
|PL PH
|2 bytes
|Address in RAM of the next line of BASIC; unsigned 16-bit integer, little-endian. PL PH equal to zero marks the end of a tokenized BASIC listing. All other values can be ignored as they are merely placeholders in the file. During program load, Tandy BASIC always recalculates PL PH. Once in memory, PL PH is used to skip to the line required for GOTO, GOSUB, and READ, thus speeding up the program.
|-
|LL LH
|2 bytes
|Line number; unsigned 16-bit integer, little-endian.
|-
|B0 … Bn
|Any number
|Sequence of tokens or ASCII characters. If a character's value is ≥ 128, then it is a token. All tokens are a single byte.
|-
|NULL
|1 byte
|NULL byte (<code>0x00</code>) to signal end of line.
|}

The end of the tokenized data is signaled by two consecutive NULLs in the location where PL PH would normally begin a new line.

=== Limits ===

* Line numbers can range from 0 to 65529 (but see variances below).
* Tokenized line length is limited to 256 characters, including the trailing NULL byte. (To do: double check this.)

=== Acceptable variances ===

While Tandy BASIC will never generate such files, it has no problem loading a tokenized program that:

* Has line numbers out of order; they will be sorted when loaded.
* Contains duplicate line numbers; previous line is discarded.
* Contains arbitrary bytes for PL PH; they are always regenerated during loading.
* Has a ^Z (ASCII 26) at the end of the file.
* Contains illegal line numbers; this can be used to store hidden binary data [https://www.mail-archive.com/m100@lists.bitchin100.com/msg16195.html according to John R. Hogerhuis].

Note that while these variances are accepted on an actual Model T device (Model 100, Tandy 200, or Tandy 102), emulators may not be able to handle such files. For example, the [https://sourceforge.net/projects/virtualt/ Virtual T] emulator (as of version 1.7 in 2022) refuses to load tokenized BASIC programs that have lines out of order, duplicate lines, or bogus values for PL PH.

=== Details for PL PH ===

(Stub. This is where the correct calculation of PL PH values will go.)

== Format documentation ==

* [https://m100.bitchin100.narkive.com/eP2uSl4J/tokenized-basic-programs-bytes-of-mystery Discussion of byte format on Bitchin 100 Mailing List]
* [http://www.club100.org/library/ups/tokens.do BASIC program to create a table of tokens]
* [https://archive.org/details/HiddenPowersOfTheTrs80Model100/page/n229/mode/2up?view=theater Hidden Powers of the TRS 80 Model 100]. Appendix B lists the BASIC token values for the Model 100. Other than the omission of 255 for a single quote remark, this reference matches what hackerb9 found programmatically for the Tandy 200.
* [http://web.archive.org/web/20220324135218/https://help.ayra.ch/trs80-reference#trs-80-model-100-basic-reference Ayra Model 100 BASIC Reference]

== Other links ==

* [https://github.com/hackerb9/tokenize/ Hackerb9's simple program for tokenizing BASIC code]
* [https://sourceforge.net/projects/virtualt/ A Tandy 200 & Model 100/102 emulator]
* [http://bitchin100.com/CloudT/ A TRS-80 Model 100 emulator that runs in a web browser]

[[Category:Microsoft]]

Tandy 200 BASIC tokenized file

2022-10-05T21:01:45Z

Hackerb9: /* File Format */ Add tokenized line table caption

{{FormatInfo
|subcat=Source code
|subcat2=Tokenized BASIC
|released=1984
}}

'''Tandy 200 BASIC''' was a version of Microsoft BASIC for the Radio Shack Tandy 200 computer. The tokenizations for the TRS-80 Model 100 and Tandy 102 appear to be identical. The NEC PC-8201/8300 format is not the same, but should be similar.

== Tokens ==

Table of tokenized BASIC keywords.

{| class="wikitable"
! title="Hexadecimal code point" | Hex
! title="Decimal code point" | Dec
! title="BASIC element the token stands for" | Token meaning
! title="Miscellany" | Notes
|-
|-
|80||128||END
|-
|81||129||FOR
|-
|82||130||NEXT
|-
|83||131||DATA
|-
|84||132||INPUT
|-
|85||133||DIM
|-
|86||134||READ
|-
|87||135||LET
|-
|88||136||GOTO
|-
|89||137||RUN
|-
|8A||138||IF
|-
|8B||139||RESTORE
|-
|8C||140||GOSUB
|-
|8D||141||RETURN
|-
|8E||142||REM
|-
|8F||143||STOP
|-
|90||144||WIDTH
|-
|91||145||ELSE||When tokenizing, the Tandy 200 always adds a colon (':') before the ELSE token (3A 91). If the user actually writes :ELSE it is tokenized as 3A 3A 91.
|-
|92||146||LINE
|-
|93||147||EDIT
|-
|94||148||ERROR
|-
|95||149||RESUME
|-
|96||150||OUT
|-
|97||151||ON
|-
|98||152||DSKO$
|-
|99||153||OPEN
|-
|9A||154||CLOSE
|-
|9B||155||LOAD||Note that LOADM is simply the token for LOAD followed by an ASCII 'M'.
|-
|9C||156||MERGE
|-QUOTE
|9D||157||FILES
|-
|9E||158||SAVE
|-
|9F||159||LFILES
|-
|A0||160||LPRINT
|-
|A1||161||DEF
|-
|A2||162||POKE
|-
|A3||163||PRINT
|-
|A4||164||CONT
|-
|A5||165||LIST
|-
|A6||166||LLIST
|-
|A7||167||CLEAR
|-
|A8||168||CLOAD
|-
|A9||169||CSAVE
|-
|AA||170||TIME$
|-
|AB||171||DATE$
|-
|AC||172||DAY$
|-
|AD||173||COM
|-
|AE||174||MDM
|-
|AF||175||KEY
|-
|B0||176||CLS
|-
|B1||177||BEEP
|-
|B2||178||SOUND
|-
|B3||179||LCOPY
|-
|B4||180||PSET
|-
|B5||181||PRESET
|-
|B6||182||MOTOR
|-
|B7||183||MAX
|-
|B8||184||POWER
|-
|B9||185||CALL
|-
|BA||186||MENU
|-
|BB||187||IPL
|-
|BC||188||NAME
|-
|BD||189||KILL
|-
|BE||190||SCREEN
|-
|BF||191||NEW
|-
|C0||192||TAB(
|-
|C1||193||TO
|-
|C2||194||USING
|-
|C3||195||VARPTR
|-
|C4||196||ERL
|-
|C5||197||ERR
|-
|C6||198||STRING$
|-
|C7||199||INSTR
|-
|C8||200||DSKI$
|-
|C9||201||INKEY$
|-
|CA||202||CSRLIN
|-
|CB||203||OFF
|-
|CC||204||HIMEM
|-
|CD||205||THEN
|-
|CE||206||NOT
|-
|CF||207||STEP
|-
|D0||208||+
|-
|D1||209||-
|-
|D2||210||*
|-
|D3||211||/
|-
|D4||212||^
|-
|D5||213||AND
|-
|D6||214||OR
|-
|D7||215||XOR
|-
|D8||216||EQV
|-
|D9||217||IMP
|-
|DA||218||MOD
|-
|DB||219||\
|-
|DC||220||>
|-
|DD||221||=
|-
|DE||222||<
|-
|DF||223||SGN
|-
|E0||224||INT
|-
|E1||225||ABS
|-
|E2||226||FRE
|-
|E3||227||INP
|-
|E4||228||LPOS
|-
|E5||229||POS
|-
|E6||230||SQR
|-
|E7||231||RND
|-
|E8||232||LOG
|-
|E9||233||EXP
|-
|EA||234||COS
|-
|EB||235||SIN
|-
|EC||236||TAN
|-
|ED||237||ATN
|-
|EE||238||PEEK
|-
|EF||239||EOF
|-
|F0||240||LOC
|-
|F1||241||LOF
|-
|F2||242||CINT
|-
|F3||243||CSNG
|-
|F4||244||CDBL
|-
|F5||245||FIX
|-
|F6||246||LEN
|-
|F7||247||STR$
|-
|F8||248||VAL
|-
|F9||249||ASC
|-
|FA||250||CHR$
|-
|FB||251||SPACE$
|-
|FC||252||LEFT$
|-
|FD||253||RIGHT$
|-
|FE||254||MID$
|-
|FF||255||' (QUOTE) || When tokenizing, the single quote character expands to three characters: a colon (3A), the byte for REM (8E), and then FF.
|}

== File Format ==

Tokenised BASIC code is a sequence of tokenized lines. Each tokenized line has the following format:

{|class="wikitable"
|+ Format of a single tokenized line
!Name!!Length!!Description
|-
|PL PH
|2 bytes
|Address in RAM of the next line of BASIC; unsigned 16-bit integer, little-endian. PL PH equal to zero marks the end of a tokenized BASIC listing. All other values can be ignored as they are merely placeholders in the file. During program load, Tandy BASIC always recalculates PL PH. Once in memory, PL PH is used to skip to the line required for GOTO, GOSUB, and READ, thus speeding up the program.
|-
|LL LH
|2 bytes
|Line number; unsigned 16-bit integer, little-endian.
|-
|B0 … Bn
|Any number
|Sequence of tokens or ASCII characters. If a character's value is ≥ 128, then it is a token. All tokens are a single byte.
|-
|NULL
|1 byte
|NULL byte (<code>0x00</code>) to signal end of line.
|}

The end of the tokenized data is signaled by two consecutive NULLs in the location where PL PH would normally begin a new line.

=== Limits ===

* Line numbers can range from 0 to 65529 (but see variances below).
* Tokenized line length is limited to 256 characters, including the trailing NULL byte. (To do: double check this.)

=== Acceptable variances ===

While Tandy BASIC will never generate such files, it has no problem loading a tokenized program that:

* Has line numbers out of order; they will be sorted when loaded.
* Contains duplicate line numbers; previous line is discarded.
* Contains arbitrary bytes for PL PH; they are always regenerated during loading.
* Has a ^Z (ASCII 26) at the end of the file.
* Contains illegal line numbers; this can be used to store hidden binary data [https://www.mail-archive.com/m100@lists.bitchin100.com/msg16195.html according to John R. Hogerhuis].

Note that while these variances are accepted on an actual Model T device (Model 100, Tandy 200, or Tandy 102), emulators may not be able to handle such files. For example, the [https://sourceforge.net/projects/virtualt/ Virtual T] emulator (as of version 1.7 in 2022) refuses to load tokenized BASIC programs that have lines out of order, duplicate lines, or bogus values for PL PH.

=== Details for PL PH ===

(Stub. This is where the correct calculation of PL PH values will go.)

== Format documentation ==

* [https://m100.bitchin100.narkive.com/eP2uSl4J/tokenized-basic-programs-bytes-of-mystery Discussion of byte format on Bitchin 100 Mailing List]
* [http://www.club100.org/library/ups/tokens.do BASIC program to create a table of tokens]
* [https://archive.org/details/HiddenPowersOfTheTrs80Model100/page/n229/mode/2up?view=theater Hidden Powers of the TRS 80 Model 100]. Appendix B lists the BASIC token values for the Model 100. Other than the omission of 255 for a single quote remark, this reference matches what hackerb9 found programmatically for the Tandy 200.
* [http://web.archive.org/web/20220324135218/https://help.ayra.ch/trs80-reference#trs-80-model-100-basic-reference Ayra Model 100 BASIC Reference]

== Other links ==

* [https://github.com/hackerb9/tokenize/ Hackerb9's simple program for tokenizing BASIC code]
* [https://sourceforge.net/projects/virtualt/ A Tandy 200 & Model 100/102 emulator]
* [http://bitchin100.com/CloudT/ A TRS-80 Model 100 emulator that runs in a web browser]

[[Category:Microsoft]]

Tandy 200 BASIC tokenized file

2022-10-05T20:56:10Z

Hackerb9: /* Acceptable variances */ use specific verb

{{FormatInfo
|subcat=Source code
|subcat2=Tokenized BASIC
|released=1984
}}

'''Tandy 200 BASIC''' was a version of Microsoft BASIC for the Radio Shack Tandy 200 computer. The tokenizations for the TRS-80 Model 100 and Tandy 102 appear to be identical. The NEC PC-8201/8300 format is not the same, but should be similar.

== Tokens ==

Table of tokenized BASIC keywords.

{| class="wikitable"
! title="Hexadecimal code point" | Hex
! title="Decimal code point" | Dec
! title="BASIC element the token stands for" | Token meaning
! title="Miscellany" | Notes
|-
|-
|80||128||END
|-
|81||129||FOR
|-
|82||130||NEXT
|-
|83||131||DATA
|-
|84||132||INPUT
|-
|85||133||DIM
|-
|86||134||READ
|-
|87||135||LET
|-
|88||136||GOTO
|-
|89||137||RUN
|-
|8A||138||IF
|-
|8B||139||RESTORE
|-
|8C||140||GOSUB
|-
|8D||141||RETURN
|-
|8E||142||REM
|-
|8F||143||STOP
|-
|90||144||WIDTH
|-
|91||145||ELSE||When tokenizing, the Tandy 200 always adds a colon (':') before the ELSE token (3A 91). If the user actually writes :ELSE it is tokenized as 3A 3A 91.
|-
|92||146||LINE
|-
|93||147||EDIT
|-
|94||148||ERROR
|-
|95||149||RESUME
|-
|96||150||OUT
|-
|97||151||ON
|-
|98||152||DSKO$
|-
|99||153||OPEN
|-
|9A||154||CLOSE
|-
|9B||155||LOAD||Note that LOADM is simply the token for LOAD followed by an ASCII 'M'.
|-
|9C||156||MERGE
|-QUOTE
|9D||157||FILES
|-
|9E||158||SAVE
|-
|9F||159||LFILES
|-
|A0||160||LPRINT
|-
|A1||161||DEF
|-
|A2||162||POKE
|-
|A3||163||PRINT
|-
|A4||164||CONT
|-
|A5||165||LIST
|-
|A6||166||LLIST
|-
|A7||167||CLEAR
|-
|A8||168||CLOAD
|-
|A9||169||CSAVE
|-
|AA||170||TIME$
|-
|AB||171||DATE$
|-
|AC||172||DAY$
|-
|AD||173||COM
|-
|AE||174||MDM
|-
|AF||175||KEY
|-
|B0||176||CLS
|-
|B1||177||BEEP
|-
|B2||178||SOUND
|-
|B3||179||LCOPY
|-
|B4||180||PSET
|-
|B5||181||PRESET
|-
|B6||182||MOTOR
|-
|B7||183||MAX
|-
|B8||184||POWER
|-
|B9||185||CALL
|-
|BA||186||MENU
|-
|BB||187||IPL
|-
|BC||188||NAME
|-
|BD||189||KILL
|-
|BE||190||SCREEN
|-
|BF||191||NEW
|-
|C0||192||TAB(
|-
|C1||193||TO
|-
|C2||194||USING
|-
|C3||195||VARPTR
|-
|C4||196||ERL
|-
|C5||197||ERR
|-
|C6||198||STRING$
|-
|C7||199||INSTR
|-
|C8||200||DSKI$
|-
|C9||201||INKEY$
|-
|CA||202||CSRLIN
|-
|CB||203||OFF
|-
|CC||204||HIMEM
|-
|CD||205||THEN
|-
|CE||206||NOT
|-
|CF||207||STEP
|-
|D0||208||+
|-
|D1||209||-
|-
|D2||210||*
|-
|D3||211||/
|-
|D4||212||^
|-
|D5||213||AND
|-
|D6||214||OR
|-
|D7||215||XOR
|-
|D8||216||EQV
|-
|D9||217||IMP
|-
|DA||218||MOD
|-
|DB||219||\
|-
|DC||220||>
|-
|DD||221||=
|-
|DE||222||<
|-
|DF||223||SGN
|-
|E0||224||INT
|-
|E1||225||ABS
|-
|E2||226||FRE
|-
|E3||227||INP
|-
|E4||228||LPOS
|-
|E5||229||POS
|-
|E6||230||SQR
|-
|E7||231||RND
|-
|E8||232||LOG
|-
|E9||233||EXP
|-
|EA||234||COS
|-
|EB||235||SIN
|-
|EC||236||TAN
|-
|ED||237||ATN
|-
|EE||238||PEEK
|-
|EF||239||EOF
|-
|F0||240||LOC
|-
|F1||241||LOF
|-
|F2||242||CINT
|-
|F3||243||CSNG
|-
|F4||244||CDBL
|-
|F5||245||FIX
|-
|F6||246||LEN
|-
|F7||247||STR$
|-
|F8||248||VAL
|-
|F9||249||ASC
|-
|FA||250||CHR$
|-
|FB||251||SPACE$
|-
|FC||252||LEFT$
|-
|FD||253||RIGHT$
|-
|FE||254||MID$
|-
|FF||255||' (QUOTE) || When tokenizing, the single quote character expands to three characters: a colon (3A), the byte for REM (8E), and then FF.
|}

== File Format ==

Tokenised BASIC code is a sequence of tokenized lines. Each tokenized line has the following format:

{|class="wikitable"
!Name!!Length!!Description
|-
|PL PH
|2 bytes
|Address in RAM of the next line of BASIC; unsigned 16-bit integer, little-endian. PL PH equal to zero marks the end of a tokenized BASIC listing. All other values can be ignored as they are merely placeholders in the file. During program load, Tandy BASIC always recalculates PL PH. Once in memory, PL PH is used to skip to the line required for GOTO, GOSUB, and READ, thus speeding up the program.
|-
|LL LH
|2 bytes
|Line number, unsigned 16-bit integer, little-endian.
|-
|B0 … Bn
|Any number
|Sequence of tokens or ASCII characters. If a character's value is ≥ 128, then it is a token. All tokens are a single byte.
|-
|NULL
|1 byte
|NULL byte (<code>0x00</code>) to signal end of line.
|}

The end of the tokenized data is signaled by two consecutive NULLs in the location where PL PH would normally begin a new line.

=== Limits ===

* Line numbers can range from 0 to 65529 (but see variances below).
* Tokenized line length is limited to 256 characters, including the trailing NULL byte. (To do: double check this.)

=== Acceptable variances ===

While Tandy BASIC will never generate such files, it has no problem loading a tokenized program that:

* Has line numbers out of order; they will be sorted when loaded.
* Contains duplicate line numbers; previous line is discarded.
* Contains arbitrary bytes for PL PH; they are always regenerated during loading.
* Has a ^Z (ASCII 26) at the end of the file.
* Contains illegal line numbers; this can be used to store hidden binary data [https://www.mail-archive.com/m100@lists.bitchin100.com/msg16195.html according to John R. Hogerhuis].

Note that while these variances are accepted on an actual Model T device (Model 100, Tandy 200, or Tandy 102), emulators may not be able to handle such files. For example, the [https://sourceforge.net/projects/virtualt/ Virtual T] emulator (as of version 1.7 in 2022) refuses to load tokenized BASIC programs that have lines out of order, duplicate lines, or bogus values for PL PH.

=== Details for PL PH ===

(Stub. This is where the correct calculation of PL PH values will go.)

== Format documentation ==

* [https://m100.bitchin100.narkive.com/eP2uSl4J/tokenized-basic-programs-bytes-of-mystery Discussion of byte format on Bitchin 100 Mailing List]
* [http://www.club100.org/library/ups/tokens.do BASIC program to create a table of tokens]
* [https://archive.org/details/HiddenPowersOfTheTrs80Model100/page/n229/mode/2up?view=theater Hidden Powers of the TRS 80 Model 100]. Appendix B lists the BASIC token values for the Model 100. Other than the omission of 255 for a single quote remark, this reference matches what hackerb9 found programmatically for the Tandy 200.
* [http://web.archive.org/web/20220324135218/https://help.ayra.ch/trs80-reference#trs-80-model-100-basic-reference Ayra Model 100 BASIC Reference]

== Other links ==

* [https://github.com/hackerb9/tokenize/ Hackerb9's simple program for tokenizing BASIC code]
* [https://sourceforge.net/projects/virtualt/ A Tandy 200 & Model 100/102 emulator]
* [http://bitchin100.com/CloudT/ A TRS-80 Model 100 emulator that runs in a web browser]

[[Category:Microsoft]]

Tandy 200 BASIC tokenized file

2022-10-05T20:53:36Z

Hackerb9: /* File Format */ Tidy language

{{FormatInfo
|subcat=Source code
|subcat2=Tokenized BASIC
|released=1984
}}

'''Tandy 200 BASIC''' was a version of Microsoft BASIC for the Radio Shack Tandy 200 computer. The tokenizations for the TRS-80 Model 100 and Tandy 102 appear to be identical. The NEC PC-8201/8300 format is not the same, but should be similar.

== Tokens ==

Table of tokenized BASIC keywords.

{| class="wikitable"
! title="Hexadecimal code point" | Hex
! title="Decimal code point" | Dec
! title="BASIC element the token stands for" | Token meaning
! title="Miscellany" | Notes
|-
|-
|80||128||END
|-
|81||129||FOR
|-
|82||130||NEXT
|-
|83||131||DATA
|-
|84||132||INPUT
|-
|85||133||DIM
|-
|86||134||READ
|-
|87||135||LET
|-
|88||136||GOTO
|-
|89||137||RUN
|-
|8A||138||IF
|-
|8B||139||RESTORE
|-
|8C||140||GOSUB
|-
|8D||141||RETURN
|-
|8E||142||REM
|-
|8F||143||STOP
|-
|90||144||WIDTH
|-
|91||145||ELSE||When tokenizing, the Tandy 200 always adds a colon (':') before the ELSE token (3A 91). If the user actually writes :ELSE it is tokenized as 3A 3A 91.
|-
|92||146||LINE
|-
|93||147||EDIT
|-
|94||148||ERROR
|-
|95||149||RESUME
|-
|96||150||OUT
|-
|97||151||ON
|-
|98||152||DSKO$
|-
|99||153||OPEN
|-
|9A||154||CLOSE
|-
|9B||155||LOAD||Note that LOADM is simply the token for LOAD followed by an ASCII 'M'.
|-
|9C||156||MERGE
|-QUOTE
|9D||157||FILES
|-
|9E||158||SAVE
|-
|9F||159||LFILES
|-
|A0||160||LPRINT
|-
|A1||161||DEF
|-
|A2||162||POKE
|-
|A3||163||PRINT
|-
|A4||164||CONT
|-
|A5||165||LIST
|-
|A6||166||LLIST
|-
|A7||167||CLEAR
|-
|A8||168||CLOAD
|-
|A9||169||CSAVE
|-
|AA||170||TIME$
|-
|AB||171||DATE$
|-
|AC||172||DAY$
|-
|AD||173||COM
|-
|AE||174||MDM
|-
|AF||175||KEY
|-
|B0||176||CLS
|-
|B1||177||BEEP
|-
|B2||178||SOUND
|-
|B3||179||LCOPY
|-
|B4||180||PSET
|-
|B5||181||PRESET
|-
|B6||182||MOTOR
|-
|B7||183||MAX
|-
|B8||184||POWER
|-
|B9||185||CALL
|-
|BA||186||MENU
|-
|BB||187||IPL
|-
|BC||188||NAME
|-
|BD||189||KILL
|-
|BE||190||SCREEN
|-
|BF||191||NEW
|-
|C0||192||TAB(
|-
|C1||193||TO
|-
|C2||194||USING
|-
|C3||195||VARPTR
|-
|C4||196||ERL
|-
|C5||197||ERR
|-
|C6||198||STRING$
|-
|C7||199||INSTR
|-
|C8||200||DSKI$
|-
|C9||201||INKEY$
|-
|CA||202||CSRLIN
|-
|CB||203||OFF
|-
|CC||204||HIMEM
|-
|CD||205||THEN
|-
|CE||206||NOT
|-
|CF||207||STEP
|-
|D0||208||+
|-
|D1||209||-
|-
|D2||210||*
|-
|D3||211||/
|-
|D4||212||^
|-
|D5||213||AND
|-
|D6||214||OR
|-
|D7||215||XOR
|-
|D8||216||EQV
|-
|D9||217||IMP
|-
|DA||218||MOD
|-
|DB||219||\
|-
|DC||220||>
|-
|DD||221||=
|-
|DE||222||<
|-
|DF||223||SGN
|-
|E0||224||INT
|-
|E1||225||ABS
|-
|E2||226||FRE
|-
|E3||227||INP
|-
|E4||228||LPOS
|-
|E5||229||POS
|-
|E6||230||SQR
|-
|E7||231||RND
|-
|E8||232||LOG
|-
|E9||233||EXP
|-
|EA||234||COS
|-
|EB||235||SIN
|-
|EC||236||TAN
|-
|ED||237||ATN
|-
|EE||238||PEEK
|-
|EF||239||EOF
|-
|F0||240||LOC
|-
|F1||241||LOF
|-
|F2||242||CINT
|-
|F3||243||CSNG
|-
|F4||244||CDBL
|-
|F5||245||FIX
|-
|F6||246||LEN
|-
|F7||247||STR$
|-
|F8||248||VAL
|-
|F9||249||ASC
|-
|FA||250||CHR$
|-
|FB||251||SPACE$
|-
|FC||252||LEFT$
|-
|FD||253||RIGHT$
|-
|FE||254||MID$
|-
|FF||255||' (QUOTE) || When tokenizing, the single quote character expands to three characters: a colon (3A), the byte for REM (8E), and then FF.
|}

== File Format ==

Tokenised BASIC code is a sequence of tokenized lines. Each tokenized line has the following format:

{|class="wikitable"
!Name!!Length!!Description
|-
|PL PH
|2 bytes
|Address in RAM of the next line of BASIC; unsigned 16-bit integer, little-endian. PL PH equal to zero marks the end of a tokenized BASIC listing. All other values can be ignored as they are merely placeholders in the file. During program load, Tandy BASIC always recalculates PL PH. Once in memory, PL PH is used to skip to the line required for GOTO, GOSUB, and READ, thus speeding up the program.
|-
|LL LH
|2 bytes
|Line number, unsigned 16-bit integer, little-endian.
|-
|B0 … Bn
|Any number
|Sequence of tokens or ASCII characters. If a character's value is ≥ 128, then it is a token. All tokens are a single byte.
|-
|NULL
|1 byte
|NULL byte (<code>0x00</code>) to signal end of line.
|}

The end of the tokenized data is signaled by two consecutive NULLs in the location where PL PH would normally begin a new line.

=== Limits ===

* Line numbers can range from 0 to 65529 (but see variances below).
* Tokenized line length is limited to 256 characters, including the trailing NULL byte. (To do: double check this.)

=== Acceptable variances ===

While Tandy BASIC will never generate such files, it has no problem with a tokenized program that:

* Has line numbers out of order; they will be sorted when loaded.
* Contains duplicate line numbers; previous line is discarded.
* Contains arbitrary bytes for PL PH; they are always regenerated during loading.
* Has a ^Z (ASCII 26) at the end of the file.
* Contains illegal line numbers; this can be used to store hidden binary data [https://www.mail-archive.com/m100@lists.bitchin100.com/msg16195.html according to John R. Hogerhuis].

Note that while these variances are accepted on an actual Model T device (Model 100, Tandy 200, or Tandy 102), emulators may not be able to handle such files. For example, the [https://sourceforge.net/projects/virtualt/ Virtual T] emulator (as of version 1.7 in 2022) refuses to load tokenized BASIC programs that have lines out of order, duplicate lines, or bogus values for PL PH.

=== Details for PL PH ===

(Stub. This is where the correct calculation of PL PH values will go.)

== Format documentation ==

* [https://m100.bitchin100.narkive.com/eP2uSl4J/tokenized-basic-programs-bytes-of-mystery Discussion of byte format on Bitchin 100 Mailing List]
* [http://www.club100.org/library/ups/tokens.do BASIC program to create a table of tokens]
* [https://archive.org/details/HiddenPowersOfTheTrs80Model100/page/n229/mode/2up?view=theater Hidden Powers of the TRS 80 Model 100]. Appendix B lists the BASIC token values for the Model 100. Other than the omission of 255 for a single quote remark, this reference matches what hackerb9 found programmatically for the Tandy 200.
* [http://web.archive.org/web/20220324135218/https://help.ayra.ch/trs80-reference#trs-80-model-100-basic-reference Ayra Model 100 BASIC Reference]

== Other links ==

* [https://github.com/hackerb9/tokenize/ Hackerb9's simple program for tokenizing BASIC code]
* [https://sourceforge.net/projects/virtualt/ A Tandy 200 & Model 100/102 emulator]
* [http://bitchin100.com/CloudT/ A TRS-80 Model 100 emulator that runs in a web browser]

[[Category:Microsoft]]

Tandy 200 BASIC tokenized file

2022-10-05T20:49:15Z

Hackerb9: /* Format documentation */ Mention the lack ' (single quote)

{{FormatInfo
|subcat=Source code
|subcat2=Tokenized BASIC
|released=1984
}}

'''Tandy 200 BASIC''' was a version of Microsoft BASIC for the Radio Shack Tandy 200 computer. The tokenizations for the TRS-80 Model 100 and Tandy 102 appear to be identical. The NEC PC-8201/8300 format is not the same, but should be similar.

== Tokens ==

Table of tokenized BASIC keywords.

{| class="wikitable"
! title="Hexadecimal code point" | Hex
! title="Decimal code point" | Dec
! title="BASIC element the token stands for" | Token meaning
! title="Miscellany" | Notes
|-
|-
|80||128||END
|-
|81||129||FOR
|-
|82||130||NEXT
|-
|83||131||DATA
|-
|84||132||INPUT
|-
|85||133||DIM
|-
|86||134||READ
|-
|87||135||LET
|-
|88||136||GOTO
|-
|89||137||RUN
|-
|8A||138||IF
|-
|8B||139||RESTORE
|-
|8C||140||GOSUB
|-
|8D||141||RETURN
|-
|8E||142||REM
|-
|8F||143||STOP
|-
|90||144||WIDTH
|-
|91||145||ELSE||When tokenizing, the Tandy 200 always adds a colon (':') before the ELSE token (3A 91). If the user actually writes :ELSE it is tokenized as 3A 3A 91.
|-
|92||146||LINE
|-
|93||147||EDIT
|-
|94||148||ERROR
|-
|95||149||RESUME
|-
|96||150||OUT
|-
|97||151||ON
|-
|98||152||DSKO$
|-
|99||153||OPEN
|-
|9A||154||CLOSE
|-
|9B||155||LOAD||Note that LOADM is simply the token for LOAD followed by an ASCII 'M'.
|-
|9C||156||MERGE
|-QUOTE
|9D||157||FILES
|-
|9E||158||SAVE
|-
|9F||159||LFILES
|-
|A0||160||LPRINT
|-
|A1||161||DEF
|-
|A2||162||POKE
|-
|A3||163||PRINT
|-
|A4||164||CONT
|-
|A5||165||LIST
|-
|A6||166||LLIST
|-
|A7||167||CLEAR
|-
|A8||168||CLOAD
|-
|A9||169||CSAVE
|-
|AA||170||TIME$
|-
|AB||171||DATE$
|-
|AC||172||DAY$
|-
|AD||173||COM
|-
|AE||174||MDM
|-
|AF||175||KEY
|-
|B0||176||CLS
|-
|B1||177||BEEP
|-
|B2||178||SOUND
|-
|B3||179||LCOPY
|-
|B4||180||PSET
|-
|B5||181||PRESET
|-
|B6||182||MOTOR
|-
|B7||183||MAX
|-
|B8||184||POWER
|-
|B9||185||CALL
|-
|BA||186||MENU
|-
|BB||187||IPL
|-
|BC||188||NAME
|-
|BD||189||KILL
|-
|BE||190||SCREEN
|-
|BF||191||NEW
|-
|C0||192||TAB(
|-
|C1||193||TO
|-
|C2||194||USING
|-
|C3||195||VARPTR
|-
|C4||196||ERL
|-
|C5||197||ERR
|-
|C6||198||STRING$
|-
|C7||199||INSTR
|-
|C8||200||DSKI$
|-
|C9||201||INKEY$
|-
|CA||202||CSRLIN
|-
|CB||203||OFF
|-
|CC||204||HIMEM
|-
|CD||205||THEN
|-
|CE||206||NOT
|-
|CF||207||STEP
|-
|D0||208||+
|-
|D1||209||-
|-
|D2||210||*
|-
|D3||211||/
|-
|D4||212||^
|-
|D5||213||AND
|-
|D6||214||OR
|-
|D7||215||XOR
|-
|D8||216||EQV
|-
|D9||217||IMP
|-
|DA||218||MOD
|-
|DB||219||\
|-
|DC||220||>
|-
|DD||221||=
|-
|DE||222||<
|-
|DF||223||SGN
|-
|E0||224||INT
|-
|E1||225||ABS
|-
|E2||226||FRE
|-
|E3||227||INP
|-
|E4||228||LPOS
|-
|E5||229||POS
|-
|E6||230||SQR
|-
|E7||231||RND
|-
|E8||232||LOG
|-
|E9||233||EXP
|-
|EA||234||COS
|-
|EB||235||SIN
|-
|EC||236||TAN
|-
|ED||237||ATN
|-
|EE||238||PEEK
|-
|EF||239||EOF
|-
|F0||240||LOC
|-
|F1||241||LOF
|-
|F2||242||CINT
|-
|F3||243||CSNG
|-
|F4||244||CDBL
|-
|F5||245||FIX
|-
|F6||246||LEN
|-
|F7||247||STR$
|-
|F8||248||VAL
|-
|F9||249||ASC
|-
|FA||250||CHR$
|-
|FB||251||SPACE$
|-
|FC||252||LEFT$
|-
|FD||253||RIGHT$
|-
|FE||254||MID$
|-
|FF||255||' (QUOTE) || When tokenizing, the single quote character expands to three characters: a colon (3A), the byte for REM (8E), and then FF.
|}

== File Format ==

Tokenised BASIC code is a sequence of tokenized lines. Each tokenized line has the following format:

{|class="wikitable"
!Name!!Length!!Description
|-
|PL PH
|2 bytes
|Address in RAM of the next line of BASIC, unsigned 16-bit integer, little-endian. The end of a tokenized BASIC listing is marked by PL PH being zero. All other values can be ignored as they are merely placeholders in the file. During program load, Tandy BASIC always recalculates PL PH. Once in memory, PL PH is used to skip to the line required for GOTO, GOSUB, and READ, thus speeding up the program.
|-
|LL LH
|2 bytes
|Line number, unsigned 16-bit integer, little-endian.
|-
|B0 … Bn
|Any number
|Sequence of tokens or ASCII characters. If a character's value is ≥ 128, then it is a token. All tokens are a single byte.
|-
|NULL
|1 byte
|NULL byte (<code>0x00</code>) to signal end of line.
|}

The end of the tokenized data is signaled by two consecutive NULLs in the location where PL PH would normally begin a new line.

=== Limits ===

* Line numbers can range from 0 to 65529 (but see variances below).
* Tokenized line length is limited to 256 characters, including the trailing NULL byte. (To do: double check this.)

=== Acceptable variances ===

While Tandy BASIC will never generate such files, it has no problem with a tokenized program that:

* Has line numbers out of order; they will be sorted when loaded.
* Contains duplicate line numbers; previous line is discarded.
* Contains arbitrary bytes for PL PH; they are always regenerated during loading.
* Has a ^Z (ASCII 26) at the end of the file.
* Contains illegal line numbers; this can be used to store hidden binary data [https://www.mail-archive.com/m100@lists.bitchin100.com/msg16195.html according to John R. Hogerhuis].

Note that while these variances are accepted on an actual Model T device (Model 100, Tandy 200, or Tandy 102), emulators may not be able to handle such files. For example, the [https://sourceforge.net/projects/virtualt/ Virtual T] emulator (as of version 1.7 in 2022) refuses to load tokenized BASIC programs that have lines out of order, duplicate lines, or bogus values for PL PH.

=== Details for PL PH ===

(Stub. This is where the correct calculation of PL PH values will go.)

== Format documentation ==

* [https://m100.bitchin100.narkive.com/eP2uSl4J/tokenized-basic-programs-bytes-of-mystery Discussion of byte format on Bitchin 100 Mailing List]
* [http://www.club100.org/library/ups/tokens.do BASIC program to create a table of tokens]
* [https://archive.org/details/HiddenPowersOfTheTrs80Model100/page/n229/mode/2up?view=theater Hidden Powers of the TRS 80 Model 100]. Appendix B lists the BASIC token values for the Model 100. Other than the omission of 255 for a single quote remark, this reference matches what hackerb9 found programmatically for the Tandy 200.
* [http://web.archive.org/web/20220324135218/https://help.ayra.ch/trs80-reference#trs-80-model-100-basic-reference Ayra Model 100 BASIC Reference]

== Other links ==

* [https://github.com/hackerb9/tokenize/ Hackerb9's simple program for tokenizing BASIC code]
* [https://sourceforge.net/projects/virtualt/ A Tandy 200 & Model 100/102 emulator]
* [http://bitchin100.com/CloudT/ A TRS-80 Model 100 emulator that runs in a web browser]

[[Category:Microsoft]]

Tandy 200 BASIC tokenized file

2022-10-05T20:43:09Z

Hackerb9: /* File Format */ Add information about the marker for the end of listing

{{FormatInfo
|subcat=Source code
|subcat2=Tokenized BASIC
|released=1984
}}

'''Tandy 200 BASIC''' was a version of Microsoft BASIC for the Radio Shack Tandy 200 computer. The tokenizations for the TRS-80 Model 100 and Tandy 102 appear to be identical. The NEC PC-8201/8300 format is not the same, but should be similar.

== Tokens ==

Table of tokenized BASIC keywords.

{| class="wikitable"
! title="Hexadecimal code point" | Hex
! title="Decimal code point" | Dec
! title="BASIC element the token stands for" | Token meaning
! title="Miscellany" | Notes
|-
|-
|80||128||END
|-
|81||129||FOR
|-
|82||130||NEXT
|-
|83||131||DATA
|-
|84||132||INPUT
|-
|85||133||DIM
|-
|86||134||READ
|-
|87||135||LET
|-
|88||136||GOTO
|-
|89||137||RUN
|-
|8A||138||IF
|-
|8B||139||RESTORE
|-
|8C||140||GOSUB
|-
|8D||141||RETURN
|-
|8E||142||REM
|-
|8F||143||STOP
|-
|90||144||WIDTH
|-
|91||145||ELSE||When tokenizing, the Tandy 200 always adds a colon (':') before the ELSE token (3A 91). If the user actually writes :ELSE it is tokenized as 3A 3A 91.
|-
|92||146||LINE
|-
|93||147||EDIT
|-
|94||148||ERROR
|-
|95||149||RESUME
|-
|96||150||OUT
|-
|97||151||ON
|-
|98||152||DSKO$
|-
|99||153||OPEN
|-
|9A||154||CLOSE
|-
|9B||155||LOAD||Note that LOADM is simply the token for LOAD followed by an ASCII 'M'.
|-
|9C||156||MERGE
|-QUOTE
|9D||157||FILES
|-
|9E||158||SAVE
|-
|9F||159||LFILES
|-
|A0||160||LPRINT
|-
|A1||161||DEF
|-
|A2||162||POKE
|-
|A3||163||PRINT
|-
|A4||164||CONT
|-
|A5||165||LIST
|-
|A6||166||LLIST
|-
|A7||167||CLEAR
|-
|A8||168||CLOAD
|-
|A9||169||CSAVE
|-
|AA||170||TIME$
|-
|AB||171||DATE$
|-
|AC||172||DAY$
|-
|AD||173||COM
|-
|AE||174||MDM
|-
|AF||175||KEY
|-
|B0||176||CLS
|-
|B1||177||BEEP
|-
|B2||178||SOUND
|-
|B3||179||LCOPY
|-
|B4||180||PSET
|-
|B5||181||PRESET
|-
|B6||182||MOTOR
|-
|B7||183||MAX
|-
|B8||184||POWER
|-
|B9||185||CALL
|-
|BA||186||MENU
|-
|BB||187||IPL
|-
|BC||188||NAME
|-
|BD||189||KILL
|-
|BE||190||SCREEN
|-
|BF||191||NEW
|-
|C0||192||TAB(
|-
|C1||193||TO
|-
|C2||194||USING
|-
|C3||195||VARPTR
|-
|C4||196||ERL
|-
|C5||197||ERR
|-
|C6||198||STRING$
|-
|C7||199||INSTR
|-
|C8||200||DSKI$
|-
|C9||201||INKEY$
|-
|CA||202||CSRLIN
|-
|CB||203||OFF
|-
|CC||204||HIMEM
|-
|CD||205||THEN
|-
|CE||206||NOT
|-
|CF||207||STEP
|-
|D0||208||+
|-
|D1||209||-
|-
|D2||210||*
|-
|D3||211||/
|-
|D4||212||^
|-
|D5||213||AND
|-
|D6||214||OR
|-
|D7||215||XOR
|-
|D8||216||EQV
|-
|D9||217||IMP
|-
|DA||218||MOD
|-
|DB||219||\
|-
|DC||220||>
|-
|DD||221||=
|-
|DE||222||<
|-
|DF||223||SGN
|-
|E0||224||INT
|-
|E1||225||ABS
|-
|E2||226||FRE
|-
|E3||227||INP
|-
|E4||228||LPOS
|-
|E5||229||POS
|-
|E6||230||SQR
|-
|E7||231||RND
|-
|E8||232||LOG
|-
|E9||233||EXP
|-
|EA||234||COS
|-
|EB||235||SIN
|-
|EC||236||TAN
|-
|ED||237||ATN
|-
|EE||238||PEEK
|-
|EF||239||EOF
|-
|F0||240||LOC
|-
|F1||241||LOF
|-
|F2||242||CINT
|-
|F3||243||CSNG
|-
|F4||244||CDBL
|-
|F5||245||FIX
|-
|F6||246||LEN
|-
|F7||247||STR$
|-
|F8||248||VAL
|-
|F9||249||ASC
|-
|FA||250||CHR$
|-
|FB||251||SPACE$
|-
|FC||252||LEFT$
|-
|FD||253||RIGHT$
|-
|FE||254||MID$
|-
|FF||255||' (QUOTE) || When tokenizing, the single quote character expands to three characters: a colon (3A), the byte for REM (8E), and then FF.
|}

== File Format ==

Tokenised BASIC code is a sequence of tokenized lines. Each tokenized line has the following format:

{|class="wikitable"
!Name!!Length!!Description
|-
|PL PH
|2 bytes
|Address in RAM of the next line of BASIC, unsigned 16-bit integer, little-endian. The end of a tokenized BASIC listing is marked by PL PH being zero. All other values can be ignored as they are merely placeholders in the file. During program load, Tandy BASIC always recalculates PL PH. Once in memory, PL PH is used to skip to the line required for GOTO, GOSUB, and READ, thus speeding up the program.
|-
|LL LH
|2 bytes
|Line number, unsigned 16-bit integer, little-endian.
|-
|B0 … Bn
|Any number
|Sequence of tokens or ASCII characters. If a character's value is ≥ 128, then it is a token. All tokens are a single byte.
|-
|NULL
|1 byte
|NULL byte (<code>0x00</code>) to signal end of line.
|}

The end of the tokenized data is signaled by two consecutive NULLs in the location where PL PH would normally begin a new line.

=== Limits ===

* Line numbers can range from 0 to 65529 (but see variances below).
* Tokenized line length is limited to 256 characters, including the trailing NULL byte. (To do: double check this.)

=== Acceptable variances ===

While Tandy BASIC will never generate such files, it has no problem with a tokenized program that:

* Has line numbers out of order; they will be sorted when loaded.
* Contains duplicate line numbers; previous line is discarded.
* Contains arbitrary bytes for PL PH; they are always regenerated during loading.
* Has a ^Z (ASCII 26) at the end of the file.
* Contains illegal line numbers; this can be used to store hidden binary data [https://www.mail-archive.com/m100@lists.bitchin100.com/msg16195.html according to John R. Hogerhuis].

Note that while these variances are accepted on an actual Model T device (Model 100, Tandy 200, or Tandy 102), emulators may not be able to handle such files. For example, the [https://sourceforge.net/projects/virtualt/ Virtual T] emulator (as of version 1.7 in 2022) refuses to load tokenized BASIC programs that have lines out of order, duplicate lines, or bogus values for PL PH.

=== Details for PL PH ===

(Stub. This is where the correct calculation of PL PH values will go.)

== Format documentation ==

* [https://m100.bitchin100.narkive.com/eP2uSl4J/tokenized-basic-programs-bytes-of-mystery Discussion of byte format on Bitchin 100 Mailing List]
* [http://www.club100.org/library/ups/tokens.do BASIC program to create a table of tokens]
* [https://archive.org/details/HiddenPowersOfTheTrs80Model100/page/n229/mode/2up?view=theater Hidden Powers of the TRS 80 Model 100]. Appendix B lists the BASIC token values for the Model 100 which matches what hackerb9 found programmatically for the Tandy 200.
* [http://web.archive.org/web/20220324135218/https://help.ayra.ch/trs80-reference#trs-80-model-100-basic-reference Ayra Model 100 BASIC Reference]

== Other links ==

* [https://github.com/hackerb9/tokenize/ Hackerb9's simple program for tokenizing BASIC code]
* [https://sourceforge.net/projects/virtualt/ A Tandy 200 & Model 100/102 emulator]
* [http://bitchin100.com/CloudT/ A TRS-80 Model 100 emulator that runs in a web browser]

[[Category:Microsoft]]

Tandy 200 BASIC tokenized file

2022-10-05T20:07:28Z

Hackerb9: /* Tokens */ Remove vestigial notice about blank entries.

{{FormatInfo
|subcat=Source code
|subcat2=Tokenized BASIC
|released=1984
}}

'''Tandy 200 BASIC''' was a version of Microsoft BASIC for the Radio Shack Tandy 200 computer. The tokenizations for the TRS-80 Model 100 and Tandy 102 appear to be identical. The NEC PC-8201/8300 format is not the same, but should be similar.

== Tokens ==

Table of tokenized BASIC keywords.

{| class="wikitable"
! title="Hexadecimal code point" | Hex
! title="Decimal code point" | Dec
! title="BASIC element the token stands for" | Token meaning
! title="Miscellany" | Notes
|-
|-
|80||128||END
|-
|81||129||FOR
|-
|82||130||NEXT
|-
|83||131||DATA
|-
|84||132||INPUT
|-
|85||133||DIM
|-
|86||134||READ
|-
|87||135||LET
|-
|88||136||GOTO
|-
|89||137||RUN
|-
|8A||138||IF
|-
|8B||139||RESTORE
|-
|8C||140||GOSUB
|-
|8D||141||RETURN
|-
|8E||142||REM
|-
|8F||143||STOP
|-
|90||144||WIDTH
|-
|91||145||ELSE||When tokenizing, the Tandy 200 always adds a colon (':') before the ELSE token (3A 91). If the user actually writes :ELSE it is tokenized as 3A 3A 91.
|-
|92||146||LINE
|-
|93||147||EDIT
|-
|94||148||ERROR
|-
|95||149||RESUME
|-
|96||150||OUT
|-
|97||151||ON
|-
|98||152||DSKO$
|-
|99||153||OPEN
|-
|9A||154||CLOSE
|-
|9B||155||LOAD||Note that LOADM is simply the token for LOAD followed by an ASCII 'M'.
|-
|9C||156||MERGE
|-QUOTE
|9D||157||FILES
|-
|9E||158||SAVE
|-
|9F||159||LFILES
|-
|A0||160||LPRINT
|-
|A1||161||DEF
|-
|A2||162||POKE
|-
|A3||163||PRINT
|-
|A4||164||CONT
|-
|A5||165||LIST
|-
|A6||166||LLIST
|-
|A7||167||CLEAR
|-
|A8||168||CLOAD
|-
|A9||169||CSAVE
|-
|AA||170||TIME$
|-
|AB||171||DATE$
|-
|AC||172||DAY$
|-
|AD||173||COM
|-
|AE||174||MDM
|-
|AF||175||KEY
|-
|B0||176||CLS
|-
|B1||177||BEEP
|-
|B2||178||SOUND
|-
|B3||179||LCOPY
|-
|B4||180||PSET
|-
|B5||181||PRESET
|-
|B6||182||MOTOR
|-
|B7||183||MAX
|-
|B8||184||POWER
|-
|B9||185||CALL
|-
|BA||186||MENU
|-
|BB||187||IPL
|-
|BC||188||NAME
|-
|BD||189||KILL
|-
|BE||190||SCREEN
|-
|BF||191||NEW
|-
|C0||192||TAB(
|-
|C1||193||TO
|-
|C2||194||USING
|-
|C3||195||VARPTR
|-
|C4||196||ERL
|-
|C5||197||ERR
|-
|C6||198||STRING$
|-
|C7||199||INSTR
|-
|C8||200||DSKI$
|-
|C9||201||INKEY$
|-
|CA||202||CSRLIN
|-
|CB||203||OFF
|-
|CC||204||HIMEM
|-
|CD||205||THEN
|-
|CE||206||NOT
|-
|CF||207||STEP
|-
|D0||208||+
|-
|D1||209||-
|-
|D2||210||*
|-
|D3||211||/
|-
|D4||212||^
|-
|D5||213||AND
|-
|D6||214||OR
|-
|D7||215||XOR
|-
|D8||216||EQV
|-
|D9||217||IMP
|-
|DA||218||MOD
|-
|DB||219||\
|-
|DC||220||>
|-
|DD||221||=
|-
|DE||222||<
|-
|DF||223||SGN
|-
|E0||224||INT
|-
|E1||225||ABS
|-
|E2||226||FRE
|-
|E3||227||INP
|-
|E4||228||LPOS
|-
|E5||229||POS
|-
|E6||230||SQR
|-
|E7||231||RND
|-
|E8||232||LOG
|-
|E9||233||EXP
|-
|EA||234||COS
|-
|EB||235||SIN
|-
|EC||236||TAN
|-
|ED||237||ATN
|-
|EE||238||PEEK
|-
|EF||239||EOF
|-
|F0||240||LOC
|-
|F1||241||LOF
|-
|F2||242||CINT
|-
|F3||243||CSNG
|-
|F4||244||CDBL
|-
|F5||245||FIX
|-
|F6||246||LEN
|-
|F7||247||STR$
|-
|F8||248||VAL
|-
|F9||249||ASC
|-
|FA||250||CHR$
|-
|FB||251||SPACE$
|-
|FC||252||LEFT$
|-
|FD||253||RIGHT$
|-
|FE||254||MID$
|-
|FF||255||' (QUOTE) || When tokenizing, the single quote character expands to three characters: a colon (3A), the byte for REM (8E), and then FF.
|}

== File Format ==

Tokenised BASIC code is a sequence of tokenized lines. Each tokenized line has the following format:

{|class="wikitable"
!Name!!Length!!Description
|-
|PL PH
|2 bytes
|Address of the next line in memory, little endian. This data can be ignored as it is only a placeholder. The value of PL PH in a file is moot because the address is recalculated every time the program loads. Once in memory, PL PH is used to skip to the line required for GOTO, GOSUB, and READ, thus speeding up the program.
|-
|LL LH
|2 bytes
|Line number, little endian.
|-
|B0 … BN
|Any number
|Sequence of tokens or ASCII characters. If a character's value is ≥ 128, then it is a token. All tokens are a single byte.
|-
|NULL
|1 byte
|NULL byte (`0x00`) to signal end of line.
|}

=== Limits ===

* Line numbers can range from 0 to 65529 (but see variances below).
* Tokenized line length is limited to 256 characters, including the trailing NULL byte. (To do: double check this.)

=== Acceptable variances ===

While Tandy BASIC will never generate such files, it has no problem with a tokenized program that:

* Has line numbers out of order; they will be sorted when loaded.
* Contains duplicate line numbers; previous line is discarded.
* Contains arbitrary bytes for PL PH; they are always regenerated during loading.
* Has a ^Z (ASCII 26) at the end of the file.
* Contains illegal line numbers; this can be used to store hidden binary data [https://www.mail-archive.com/m100@lists.bitchin100.com/msg16195.html according to John R. Hogerhuis].

Note that while these variances are accepted on an actual Model T device (Model 100, Tandy 200, or Tandy 102), emulators may not be able to handle such files. For example, the [https://sourceforge.net/projects/virtualt/ Virtual T] emulator (as of version 1.7 in 2022) refuses to load tokenized BASIC programs that have lines out of order, duplicate lines, or bogus values for PL PH.

=== Details for PL PH ===

(Stub. This is where the correct calculation of PL PH values will go.)

== Format documentation ==

* [https://m100.bitchin100.narkive.com/eP2uSl4J/tokenized-basic-programs-bytes-of-mystery Discussion of byte format on Bitchin 100 Mailing List]
* [http://www.club100.org/library/ups/tokens.do BASIC program to create a table of tokens]
* [https://archive.org/details/HiddenPowersOfTheTrs80Model100/page/n229/mode/2up?view=theater Hidden Powers of the TRS 80 Model 100]. Appendix B lists the BASIC token values for the Model 100 which matches what hackerb9 found programmatically for the Tandy 200.
* [http://web.archive.org/web/20220324135218/https://help.ayra.ch/trs80-reference#trs-80-model-100-basic-reference Ayra Model 100 BASIC Reference]

== Other links ==

* [https://github.com/hackerb9/tokenize/ Hackerb9's simple program for tokenizing BASIC code]
* [https://sourceforge.net/projects/virtualt/ A Tandy 200 & Model 100/102 emulator]
* [http://bitchin100.com/CloudT/ A TRS-80 Model 100 emulator that runs in a web browser]

[[Category:Microsoft]]

Tandy 200 BASIC tokenized file

2022-10-05T20:01:48Z

Hackerb9: /* Tokens */ Actually, the problem was the final pipe symbol confused wiki table. Hopefully this fixes it.

{{FormatInfo
|subcat=Source code
|subcat2=Tokenized BASIC
|released=1984
}}

'''Tandy 200 BASIC''' was a version of Microsoft BASIC for the Radio Shack Tandy 200 computer. The tokenizations for the TRS-80 Model 100 and Tandy 102 appear to be identical. The NEC PC-8201/8300 format is not the same, but should be similar.

== Tokens ==

Blank values indicate either that the token is unused or is used for something unknown.

{| class="wikitable"
! title="Hexadecimal code point" | Hex
! title="Decimal code point" | Dec
! title="BASIC element the token stands for" | Token meaning
! title="Miscellany" | Notes
|-
|-
|80||128||END
|-
|81||129||FOR
|-
|82||130||NEXT
|-
|83||131||DATA
|-
|84||132||INPUT
|-
|85||133||DIM
|-
|86||134||READ
|-
|87||135||LET
|-
|88||136||GOTO
|-
|89||137||RUN
|-
|8A||138||IF
|-
|8B||139||RESTORE
|-
|8C||140||GOSUB
|-
|8D||141||RETURN
|-
|8E||142||REM
|-
|8F||143||STOP
|-
|90||144||WIDTH
|-
|91||145||ELSE||When tokenizing, the Tandy 200 always adds a colon (':') before the ELSE token (3A 91). If the user actually writes :ELSE it is tokenized as 3A 3A 91.
|-
|92||146||LINE
|-
|93||147||EDIT
|-
|94||148||ERROR
|-
|95||149||RESUME
|-
|96||150||OUT
|-
|97||151||ON
|-
|98||152||DSKO$
|-
|99||153||OPEN
|-
|9A||154||CLOSE
|-
|9B||155||LOAD||Note that LOADM is simply the token for LOAD followed by an ASCII 'M'.
|-
|9C||156||MERGE
|-QUOTE
|9D||157||FILES
|-
|9E||158||SAVE
|-
|9F||159||LFILES
|-
|A0||160||LPRINT
|-
|A1||161||DEF
|-
|A2||162||POKE
|-
|A3||163||PRINT
|-
|A4||164||CONT
|-
|A5||165||LIST
|-
|A6||166||LLIST
|-
|A7||167||CLEAR
|-
|A8||168||CLOAD
|-
|A9||169||CSAVE
|-
|AA||170||TIME$
|-
|AB||171||DATE$
|-
|AC||172||DAY$
|-
|AD||173||COM
|-
|AE||174||MDM
|-
|AF||175||KEY
|-
|B0||176||CLS
|-
|B1||177||BEEP
|-
|B2||178||SOUND
|-
|B3||179||LCOPY
|-
|B4||180||PSET
|-
|B5||181||PRESET
|-
|B6||182||MOTOR
|-
|B7||183||MAX
|-
|B8||184||POWER
|-
|B9||185||CALL
|-
|BA||186||MENU
|-
|BB||187||IPL
|-
|BC||188||NAME
|-
|BD||189||KILL
|-
|BE||190||SCREEN
|-
|BF||191||NEW
|-
|C0||192||TAB(
|-
|C1||193||TO
|-
|C2||194||USING
|-
|C3||195||VARPTR
|-
|C4||196||ERL
|-
|C5||197||ERR
|-
|C6||198||STRING$
|-
|C7||199||INSTR
|-
|C8||200||DSKI$
|-
|C9||201||INKEY$
|-
|CA||202||CSRLIN
|-
|CB||203||OFF
|-
|CC||204||HIMEM
|-
|CD||205||THEN
|-
|CE||206||NOT
|-
|CF||207||STEP
|-
|D0||208||+
|-
|D1||209||-
|-
|D2||210||*
|-
|D3||211||/
|-
|D4||212||^
|-
|D5||213||AND
|-
|D6||214||OR
|-
|D7||215||XOR
|-
|D8||216||EQV
|-
|D9||217||IMP
|-
|DA||218||MOD
|-
|DB||219||\
|-
|DC||220||>
|-
|DD||221||=
|-
|DE||222||<
|-
|DF||223||SGN
|-
|E0||224||INT
|-
|E1||225||ABS
|-
|E2||226||FRE
|-
|E3||227||INP
|-
|E4||228||LPOS
|-
|E5||229||POS
|-
|E6||230||SQR
|-
|E7||231||RND
|-
|E8||232||LOG
|-
|E9||233||EXP
|-
|EA||234||COS
|-
|EB||235||SIN
|-
|EC||236||TAN
|-
|ED||237||ATN
|-
|EE||238||PEEK
|-
|EF||239||EOF
|-
|F0||240||LOC
|-
|F1||241||LOF
|-
|F2||242||CINT
|-
|F3||243||CSNG
|-
|F4||244||CDBL
|-
|F5||245||FIX
|-
|F6||246||LEN
|-
|F7||247||STR$
|-
|F8||248||VAL
|-
|F9||249||ASC
|-
|FA||250||CHR$
|-
|FB||251||SPACE$
|-
|FC||252||LEFT$
|-
|FD||253||RIGHT$
|-
|FE||254||MID$
|-
|FF||255||' (QUOTE) || When tokenizing, the single quote character expands to three characters: a colon (3A), the byte for REM (8E), and then FF.
|}

== File Format ==

Tokenised BASIC code is a sequence of tokenized lines. Each tokenized line has the following format:

{|class="wikitable"
!Name!!Length!!Description
|-
|PL PH
|2 bytes
|Address of the next line in memory, little endian. This data can be ignored as it is only a placeholder. The value of PL PH in a file is moot because the address is recalculated every time the program loads. Once in memory, PL PH is used to skip to the line required for GOTO, GOSUB, and READ, thus speeding up the program.
|-
|LL LH
|2 bytes
|Line number, little endian.
|-
|B0 … BN
|Any number
|Sequence of tokens or ASCII characters. If a character's value is ≥ 128, then it is a token. All tokens are a single byte.
|-
|NULL
|1 byte
|NULL byte (`0x00`) to signal end of line.
|}

=== Limits ===

* Line numbers can range from 0 to 65529 (but see variances below).
* Tokenized line length is limited to 256 characters, including the trailing NULL byte. (To do: double check this.)

=== Acceptable variances ===

While Tandy BASIC will never generate such files, it has no problem with a tokenized program that:

* Has line numbers out of order; they will be sorted when loaded.
* Contains duplicate line numbers; previous line is discarded.
* Contains arbitrary bytes for PL PH; they are always regenerated during loading.
* Has a ^Z (ASCII 26) at the end of the file.
* Contains illegal line numbers; this can be used to store hidden binary data [https://www.mail-archive.com/m100@lists.bitchin100.com/msg16195.html according to John R. Hogerhuis].

Note that while these variances are accepted on an actual Model T device (Model 100, Tandy 200, or Tandy 102), emulators may not be able to handle such files. For example, the [https://sourceforge.net/projects/virtualt/ Virtual T] emulator (as of version 1.7 in 2022) refuses to load tokenized BASIC programs that have lines out of order, duplicate lines, or bogus values for PL PH.

=== Details for PL PH ===

(Stub. This is where the correct calculation of PL PH values will go.)

== Format documentation ==

* [https://m100.bitchin100.narkive.com/eP2uSl4J/tokenized-basic-programs-bytes-of-mystery Discussion of byte format on Bitchin 100 Mailing List]
* [http://www.club100.org/library/ups/tokens.do BASIC program to create a table of tokens]
* [https://archive.org/details/HiddenPowersOfTheTrs80Model100/page/n229/mode/2up?view=theater Hidden Powers of the TRS 80 Model 100]. Appendix B lists the BASIC token values for the Model 100 which matches what hackerb9 found programmatically for the Tandy 200.
* [http://web.archive.org/web/20220324135218/https://help.ayra.ch/trs80-reference#trs-80-model-100-basic-reference Ayra Model 100 BASIC Reference]

== Other links ==

* [https://github.com/hackerb9/tokenize/ Hackerb9's simple program for tokenizing BASIC code]
* [https://sourceforge.net/projects/virtualt/ A Tandy 200 & Model 100/102 emulator]
* [http://bitchin100.com/CloudT/ A TRS-80 Model 100 emulator that runs in a web browser]

[[Category:Microsoft]]

Tandy 200 BASIC tokenized file

2022-10-05T19:57:34Z

Hackerb9: /* Tokens */ Replace" \<" with "<" as it is resilient to repeated wiki editing

{{FormatInfo
|subcat=Source code
|subcat2=Tokenized BASIC
|released=1984
}}

'''Tandy 200 BASIC''' was a version of Microsoft BASIC for the Radio Shack Tandy 200 computer. The tokenizations for the TRS-80 Model 100 and Tandy 102 appear to be identical. The NEC PC-8201/8300 format is not the same, but should be similar.

== Tokens ==

Blank values indicate either that the token is unused or is used for something unknown.

{| class="wikitable"
! title="Hexadecimal code point" | Hex
! title="Decimal code point" | Dec
! title="BASIC element the token stands for" | Token meaning
! title="Miscellany" | Notes
|-
|-
|80||128||END
|-
|81||129||FOR
|-
|82||130||NEXT
|-
|83||131||DATA
|-
|84||132||INPUT
|-
|85||133||DIM
|-
|86||134||READ
|-
|87||135||LET
|-
|88||136||GOTO
|-
|89||137||RUN
|-
|8A||138||IF
|-
|8B||139||RESTORE
|-
|8C||140||GOSUB
|-
|8D||141||RETURN
|-
|8E||142||REM
|-
|8F||143||STOP
|-
|90||144||WIDTH
|-
|91||145||ELSE||When tokenizing, the Tandy 200 always adds a colon (':') before the ELSE token (3A 91). If the user actually writes :ELSE it is tokenized as 3A 3A 91.
|-
|92||146||LINE
|-
|93||147||EDIT
|-
|94||148||ERROR
|-
|95||149||RESUME
|-
|96||150||OUT
|-
|97||151||ON
|-
|98||152||DSKO$
|-
|99||153||OPEN
|-
|9A||154||CLOSE
|-
|9B||155||LOAD||Note that LOADM is simply the token for LOAD followed by an ASCII 'M'.
|-
|9C||156||MERGE
|-QUOTE
|9D||157||FILES
|-
|9E||158||SAVE
|-
|9F||159||LFILES
|-
|A0||160||LPRINT
|-
|A1||161||DEF
|-
|A2||162||POKE
|-
|A3||163||PRINT
|-
|A4||164||CONT
|-
|A5||165||LIST
|-
|A6||166||LLIST
|-
|A7||167||CLEAR
|-
|A8||168||CLOAD
|-
|A9||169||CSAVE
|-
|AA||170||TIME$
|-
|AB||171||DATE$
|-
|AC||172||DAY$
|-
|AD||173||COM
|-
|AE||174||MDM
|-
|AF||175||KEY
|-
|B0||176||CLS
|-
|B1||177||BEEP
|-
|B2||178||SOUND
|-
|B3||179||LCOPY
|-
|B4||180||PSET
|-
|B5||181||PRESET
|-
|B6||182||MOTOR
|-
|B7||183||MAX
|-
|B8||184||POWER
|-
|B9||185||CALL
|-
|BA||186||MENU
|-
|BB||187||IPL
|-
|BC||188||NAME
|-
|BD||189||KILL
|-
|BE||190||SCREEN
|-
|BF||191||NEW
|-
|C0||192||TAB(
|-
|C1||193||TO
|-
|C2||194||USING
|-
|C3||195||VARPTR
|-
|C4||196||ERL
|-
|C5||197||ERR
|-
|C6||198||STRING$
|-
|C7||199||INSTR
|-
|C8||200||DSKI$
|-
|C9||201||INKEY$
|-
|CA||202||CSRLIN
|-
|CB||203||OFF
|-
|CC||204||HIMEM
|-
|CD||205||THEN
|-
|CE||206||NOT
|-
|CF||207||STEP
|-
|D0||208||+
|-
|D1||209||-
|-
|D2||210||*
|-
|D3||211||/
|-
|D4||212||^
|-
|D5||213||AND
|-
|D6||214||OR
|-
|D7||215||XOR
|-
|D8||216||EQV
|-
|D9||217||IMP
|-
|DA||218||MOD
|-
|DB||219||\
|-
|DC||220||>
|-
|DD||221||=
|-
|DE||222||<|
|-
|DF||223||SGN
|-
|E0||224||INT
|-
|E1||225||ABS
|-
|E2||226||FRE
|-
|E3||227||INP
|-
|E4||228||LPOS
|-
|E5||229||POS
|-
|E6||230||SQR
|-
|E7||231||RND
|-
|E8||232||LOG
|-
|E9||233||EXP
|-
|EA||234||COS
|-
|EB||235||SIN
|-
|EC||236||TAN
|-
|ED||237||ATN
|-
|EE||238||PEEK
|-
|EF||239||EOF
|-
|F0||240||LOC
|-
|F1||241||LOF
|-
|F2||242||CINT
|-
|F3||243||CSNG
|-
|F4||244||CDBL
|-
|F5||245||FIX
|-
|F6||246||LEN
|-
|F7||247||STR$
|-
|F8||248||VAL
|-
|F9||249||ASC
|-
|FA||250||CHR$
|-
|FB||251||SPACE$
|-
|FC||252||LEFT$
|-
|FD||253||RIGHT$
|-
|FE||254||MID$
|-
|FF||255||' (QUOTE) || When tokenizing, the single quote character expands to three characters: a colon (3A), the byte for REM (8E), and then FF.
|}

== File Format ==

Tokenised BASIC code is a sequence of tokenized lines. Each tokenized line has the following format:

{|class="wikitable"
!Name!!Length!!Description
|-
|PL PH
|2 bytes
|Address of the next line in memory, little endian. This data can be ignored as it is only a placeholder. The value of PL PH in a file is moot because the address is recalculated every time the program loads. Once in memory, PL PH is used to skip to the line required for GOTO, GOSUB, and READ, thus speeding up the program.
|-
|LL LH
|2 bytes
|Line number, little endian.
|-
|B0 … BN
|Any number
|Sequence of tokens or ASCII characters. If a character's value is ≥ 128, then it is a token. All tokens are a single byte.
|-
|NULL
|1 byte
|NULL byte (`0x00`) to signal end of line.
|}

=== Limits ===

* Line numbers can range from 0 to 65529 (but see variances below).
* Tokenized line length is limited to 256 characters, including the trailing NULL byte. (To do: double check this.)

=== Acceptable variances ===

While Tandy BASIC will never generate such files, it has no problem with a tokenized program that:

* Has line numbers out of order; they will be sorted when loaded.
* Contains duplicate line numbers; previous line is discarded.
* Contains arbitrary bytes for PL PH; they are always regenerated during loading.
* Has a ^Z (ASCII 26) at the end of the file.
* Contains illegal line numbers; this can be used to store hidden binary data [https://www.mail-archive.com/m100@lists.bitchin100.com/msg16195.html according to John R. Hogerhuis].

Note that while these variances are accepted on an actual Model T device (Model 100, Tandy 200, or Tandy 102), emulators may not be able to handle such files. For example, the [https://sourceforge.net/projects/virtualt/ Virtual T] emulator (as of version 1.7 in 2022) refuses to load tokenized BASIC programs that have lines out of order, duplicate lines, or bogus values for PL PH.

=== Details for PL PH ===

(Stub. This is where the correct calculation of PL PH values will go.)

== Format documentation ==

* [https://m100.bitchin100.narkive.com/eP2uSl4J/tokenized-basic-programs-bytes-of-mystery Discussion of byte format on Bitchin 100 Mailing List]
* [http://www.club100.org/library/ups/tokens.do BASIC program to create a table of tokens]
* [https://archive.org/details/HiddenPowersOfTheTrs80Model100/page/n229/mode/2up?view=theater Hidden Powers of the TRS 80 Model 100]. Appendix B lists the BASIC token values for the Model 100 which matches what hackerb9 found programmatically for the Tandy 200.
* [http://web.archive.org/web/20220324135218/https://help.ayra.ch/trs80-reference#trs-80-model-100-basic-reference Ayra Model 100 BASIC Reference]

== Other links ==

* [https://github.com/hackerb9/tokenize/ Hackerb9's simple program for tokenizing BASIC code]
* [https://sourceforge.net/projects/virtualt/ A Tandy 200 & Model 100/102 emulator]
* [http://bitchin100.com/CloudT/ A TRS-80 Model 100 emulator that runs in a web browser]

[[Category:Microsoft]]

Tandy 200 BASIC tokenized file

2022-10-05T19:52:55Z

Hackerb9: /* Format documentation */ Add links for Hidden Powers and Ayra

{{FormatInfo
|subcat=Source code
|subcat2=Tokenized BASIC
|released=1984
}}

'''Tandy 200 BASIC''' was a version of Microsoft BASIC for the Radio Shack Tandy 200 computer. The tokenizations for the TRS-80 Model 100 and Tandy 102 appear to be identical. The NEC PC-8201/8300 format is not the same, but should be similar.

== Tokens ==

Blank values indicate either that the token is unused or is used for something unknown.

{| class="wikitable"
! title="Hexadecimal code point" | Hex
! title="Decimal code point" | Dec
! title="BASIC element the token stands for" | Token meaning
! title="Miscellany" | Notes
|-
|-
|80||128||END
|-
|81||129||FOR
|-
|82||130||NEXT
|-
|83||131||DATA
|-
|84||132||INPUT
|-
|85||133||DIM
|-
|86||134||READ
|-
|87||135||LET
|-
|88||136||GOTO
|-
|89||137||RUN
|-
|8A||138||IF
|-
|8B||139||RESTORE
|-
|8C||140||GOSUB
|-
|8D||141||RETURN
|-
|8E||142||REM
|-
|8F||143||STOP
|-
|90||144||WIDTH
|-
|91||145||ELSE||When tokenizing, the Tandy 200 always adds a colon (':') before the ELSE token (3A 91). If the user actually writes :ELSE it is tokenized as 3A 3A 91.
|-
|92||146||LINE
|-
|93||147||EDIT
|-
|94||148||ERROR
|-
|95||149||RESUME
|-
|96||150||OUT
|-
|97||151||ON
|-
|98||152||DSKO$
|-
|99||153||OPEN
|-
|9A||154||CLOSE
|-
|9B||155||LOAD||Note that LOADM is simply the token for LOAD followed by an ASCII 'M'.
|-
|9C||156||MERGE
|-QUOTE
|9D||157||FILES
|-
|9E||158||SAVE
|-
|9F||159||LFILES
|-
|A0||160||LPRINT
|-
|A1||161||DEF
|-
|A2||162||POKE
|-
|A3||163||PRINT
|-
|A4||164||CONT
|-
|A5||165||LIST
|-
|A6||166||LLIST
|-
|A7||167||CLEAR
|-
|A8||168||CLOAD
|-
|A9||169||CSAVE
|-
|AA||170||TIME$
|-
|AB||171||DATE$
|-
|AC||172||DAY$
|-
|AD||173||COM
|-
|AE||174||MDM
|-
|AF||175||KEY
|-
|B0||176||CLS
|-
|B1||177||BEEP
|-
|B2||178||SOUND
|-
|B3||179||LCOPY
|-
|B4||180||PSET
|-
|B5||181||PRESET
|-
|B6||182||MOTOR
|-
|B7||183||MAX
|-
|B8||184||POWER
|-
|B9||185||CALL
|-
|BA||186||MENU
|-
|BB||187||IPL
|-
|BC||188||NAME
|-
|BD||189||KILL
|-
|BE||190||SCREEN
|-
|BF||191||NEW
|-
|C0||192||TAB(
|-
|C1||193||TO
|-
|C2||194||USING
|-
|C3||195||VARPTR
|-
|C4||196||ERL
|-
|C5||197||ERR
|-
|C6||198||STRING$
|-
|C7||199||INSTR
|-
|C8||200||DSKI$
|-
|C9||201||INKEY$
|-
|CA||202||CSRLIN
|-
|CB||203||OFF
|-
|CC||204||HIMEM
|-
|CD||205||THEN
|-
|CE||206||NOT
|-
|CF||207||STEP
|-
|D0||208||+
|-
|D1||209||-
|-
|D2||210||*
|-
|D3||211||/
|-
|D4||212||^
|-
|D5||213||AND
|-
|D6||214||OR
|-
|D7||215||XOR
|-
|D8||216||EQV
|-
|D9||217||IMP
|-
|DA||218||MOD
|-
|DB||219||\
|-
|DC||220||>
|-
|DD||221||=
|-
|DE||222||\|
|-
|DF||223||SGN
|-
|E0||224||INT
|-
|E1||225||ABS
|-
|E2||226||FRE
|-
|E3||227||INP
|-
|E4||228||LPOS
|-
|E5||229||POS
|-
|E6||230||SQR
|-
|E7||231||RND
|-
|E8||232||LOG
|-
|E9||233||EXP
|-
|EA||234||COS
|-
|EB||235||SIN
|-
|EC||236||TAN
|-
|ED||237||ATN
|-
|EE||238||PEEK
|-
|EF||239||EOF
|-
|F0||240||LOC
|-
|F1||241||LOF
|-
|F2||242||CINT
|-
|F3||243||CSNG
|-
|F4||244||CDBL
|-
|F5||245||FIX
|-
|F6||246||LEN
|-
|F7||247||STR$
|-
|F8||248||VAL
|-
|F9||249||ASC
|-
|FA||250||CHR$
|-
|FB||251||SPACE$
|-
|FC||252||LEFT$
|-
|FD||253||RIGHT$
|-
|FE||254||MID$
|-
|FF||255||' (QUOTE) || When tokenizing, the single quote character expands to three characters: a colon (3A), the byte for REM (8E), and then FF.
|}

== File Format ==

Tokenised BASIC code is a sequence of tokenized lines. Each tokenized line has the following format:

{|class="wikitable"
!Name!!Length!!Description
|-
|PL PH
|2 bytes
|Address of the next line in memory, little endian. This data can be ignored as it is only a placeholder. The value of PL PH in a file is moot because the address is recalculated every time the program loads. Once in memory, PL PH is used to skip to the line required for GOTO, GOSUB, and READ, thus speeding up the program.
|-
|LL LH
|2 bytes
|Line number, little endian.
|-
|B0 … BN
|Any number
|Sequence of tokens or ASCII characters. If a character's value is ≥ 128, then it is a token. All tokens are a single byte.
|-
|NULL
|1 byte
|NULL byte (`0x00`) to signal end of line.
|}

=== Limits ===

* Line numbers can range from 0 to 65529 (but see variances below).
* Tokenized line length is limited to 256 characters, including the trailing NULL byte. (To do: double check this.)

=== Acceptable variances ===

While Tandy BASIC will never generate such files, it has no problem with a tokenized program that:

* Has line numbers out of order; they will be sorted when loaded.
* Contains duplicate line numbers; previous line is discarded.
* Contains arbitrary bytes for PL PH; they are always regenerated during loading.
* Has a ^Z (ASCII 26) at the end of the file.
* Contains illegal line numbers; this can be used to store hidden binary data [https://www.mail-archive.com/m100@lists.bitchin100.com/msg16195.html according to John R. Hogerhuis].

Note that while these variances are accepted on an actual Model T device (Model 100, Tandy 200, or Tandy 102), emulators may not be able to handle such files. For example, the [https://sourceforge.net/projects/virtualt/ Virtual T] emulator (as of version 1.7 in 2022) refuses to load tokenized BASIC programs that have lines out of order, duplicate lines, or bogus values for PL PH.

=== Details for PL PH ===

(Stub. This is where the correct calculation of PL PH values will go.)

== Format documentation ==

* [https://m100.bitchin100.narkive.com/eP2uSl4J/tokenized-basic-programs-bytes-of-mystery Discussion of byte format on Bitchin 100 Mailing List]
* [http://www.club100.org/library/ups/tokens.do BASIC program to create a table of tokens]
* [https://archive.org/details/HiddenPowersOfTheTrs80Model100/page/n229/mode/2up?view=theater Hidden Powers of the TRS 80 Model 100]. Appendix B lists the BASIC token values for the Model 100 which matches what hackerb9 found programmatically for the Tandy 200.
* [http://web.archive.org/web/20220324135218/https://help.ayra.ch/trs80-reference#trs-80-model-100-basic-reference Ayra Model 100 BASIC Reference]

== Other links ==

* [https://github.com/hackerb9/tokenize/ Hackerb9's simple program for tokenizing BASIC code]
* [https://sourceforge.net/projects/virtualt/ A Tandy 200 & Model 100/102 emulator]
* [http://bitchin100.com/CloudT/ A TRS-80 Model 100 emulator that runs in a web browser]

[[Category:Microsoft]]

Tandy 200 BASIC tokenized file

2022-10-05T19:35:39Z

Hackerb9: Create subsections for limits and acceptable variances

{{FormatInfo
|subcat=Source code
|subcat2=Tokenized BASIC
|released=1984
}}

'''Tandy 200 BASIC''' was a version of Microsoft BASIC for the Radio Shack Tandy 200 computer. The tokenizations for the TRS-80 Model 100 and Tandy 102 appear to be identical. The NEC PC-8201/8300 format is not the same, but should be similar.

== Tokens ==

Blank values indicate either that the token is unused or is used for something unknown.

{| class="wikitable"
! title="Hexadecimal code point" | Hex
! title="Decimal code point" | Dec
! title="BASIC element the token stands for" | Token meaning
! title="Miscellany" | Notes
|-
|-
|80||128||END
|-
|81||129||FOR
|-
|82||130||NEXT
|-
|83||131||DATA
|-
|84||132||INPUT
|-
|85||133||DIM
|-
|86||134||READ
|-
|87||135||LET
|-
|88||136||GOTO
|-
|89||137||RUN
|-
|8A||138||IF
|-
|8B||139||RESTORE
|-
|8C||140||GOSUB
|-
|8D||141||RETURN
|-
|8E||142||REM
|-
|8F||143||STOP
|-
|90||144||WIDTH
|-
|91||145||ELSE||When tokenizing, the Tandy 200 always adds a colon (':') before the ELSE token (3A 91). If the user actually writes :ELSE it is tokenized as 3A 3A 91.
|-
|92||146||LINE
|-
|93||147||EDIT
|-
|94||148||ERROR
|-
|95||149||RESUME
|-
|96||150||OUT
|-
|97||151||ON
|-
|98||152||DSKO$
|-
|99||153||OPEN
|-
|9A||154||CLOSE
|-
|9B||155||LOAD||Note that LOADM is simply the token for LOAD followed by an ASCII 'M'.
|-
|9C||156||MERGE
|-QUOTE
|9D||157||FILES
|-
|9E||158||SAVE
|-
|9F||159||LFILES
|-
|A0||160||LPRINT
|-
|A1||161||DEF
|-
|A2||162||POKE
|-
|A3||163||PRINT
|-
|A4||164||CONT
|-
|A5||165||LIST
|-
|A6||166||LLIST
|-
|A7||167||CLEAR
|-
|A8||168||CLOAD
|-
|A9||169||CSAVE
|-
|AA||170||TIME$
|-
|AB||171||DATE$
|-
|AC||172||DAY$
|-
|AD||173||COM
|-
|AE||174||MDM
|-
|AF||175||KEY
|-
|B0||176||CLS
|-
|B1||177||BEEP
|-
|B2||178||SOUND
|-
|B3||179||LCOPY
|-
|B4||180||PSET
|-
|B5||181||PRESET
|-
|B6||182||MOTOR
|-
|B7||183||MAX
|-
|B8||184||POWER
|-
|B9||185||CALL
|-
|BA||186||MENU
|-
|BB||187||IPL
|-
|BC||188||NAME
|-
|BD||189||KILL
|-
|BE||190||SCREEN
|-
|BF||191||NEW
|-
|C0||192||TAB(
|-
|C1||193||TO
|-
|C2||194||USING
|-
|C3||195||VARPTR
|-
|C4||196||ERL
|-
|C5||197||ERR
|-
|C6||198||STRING$
|-
|C7||199||INSTR
|-
|C8||200||DSKI$
|-
|C9||201||INKEY$
|-
|CA||202||CSRLIN
|-
|CB||203||OFF
|-
|CC||204||HIMEM
|-
|CD||205||THEN
|-
|CE||206||NOT
|-
|CF||207||STEP
|-
|D0||208||+
|-
|D1||209||-
|-
|D2||210||*
|-
|D3||211||/
|-
|D4||212||^
|-
|D5||213||AND
|-
|D6||214||OR
|-
|D7||215||XOR
|-
|D8||216||EQV
|-
|D9||217||IMP
|-
|DA||218||MOD
|-
|DB||219||\
|-
|DC||220||>
|-
|DD||221||=
|-
|DE||222||\|
|-
|DF||223||SGN
|-
|E0||224||INT
|-
|E1||225||ABS
|-
|E2||226||FRE
|-
|E3||227||INP
|-
|E4||228||LPOS
|-
|E5||229||POS
|-
|E6||230||SQR
|-
|E7||231||RND
|-
|E8||232||LOG
|-
|E9||233||EXP
|-
|EA||234||COS
|-
|EB||235||SIN
|-
|EC||236||TAN
|-
|ED||237||ATN
|-
|EE||238||PEEK
|-
|EF||239||EOF
|-
|F0||240||LOC
|-
|F1||241||LOF
|-
|F2||242||CINT
|-
|F3||243||CSNG
|-
|F4||244||CDBL
|-
|F5||245||FIX
|-
|F6||246||LEN
|-
|F7||247||STR$
|-
|F8||248||VAL
|-
|F9||249||ASC
|-
|FA||250||CHR$
|-
|FB||251||SPACE$
|-
|FC||252||LEFT$
|-
|FD||253||RIGHT$
|-
|FE||254||MID$
|-
|FF||255||' (QUOTE) || When tokenizing, the single quote character expands to three characters: a colon (3A), the byte for REM (8E), and then FF.
|}

== File Format ==

Tokenised BASIC code is a sequence of tokenized lines. Each tokenized line has the following format:

{|class="wikitable"
!Name!!Length!!Description
|-
|PL PH
|2 bytes
|Address of the next line in memory, little endian. This data can be ignored as it is only a placeholder. The value of PL PH in a file is moot because the address is recalculated every time the program loads. Once in memory, PL PH is used to skip to the line required for GOTO, GOSUB, and READ, thus speeding up the program.
|-
|LL LH
|2 bytes
|Line number, little endian.
|-
|B0 … BN
|Any number
|Sequence of tokens or ASCII characters. If a character's value is ≥ 128, then it is a token. All tokens are a single byte.
|-
|NULL
|1 byte
|NULL byte (`0x00`) to signal end of line.
|}

=== Limits ===

* Line numbers can range from 0 to 65529 (but see variances below).
* Tokenized line length is limited to 256 characters, including the trailing NULL byte. (To do: double check this.)

=== Acceptable variances ===

While Tandy BASIC will never generate such files, it has no problem with a tokenized program that:

* Has line numbers out of order; they will be sorted when loaded.
* Contains duplicate line numbers; previous line is discarded.
* Contains arbitrary bytes for PL PH; they are always regenerated during loading.
* Has a ^Z (ASCII 26) at the end of the file.
* Contains illegal line numbers; this can be used to store hidden binary data [https://www.mail-archive.com/m100@lists.bitchin100.com/msg16195.html according to John R. Hogerhuis].

Note that while these variances are accepted on an actual Model T device (Model 100, Tandy 200, or Tandy 102), emulators may not be able to handle such files. For example, the [https://sourceforge.net/projects/virtualt/ Virtual T] emulator (as of version 1.7 in 2022) refuses to load tokenized BASIC programs that have lines out of order, duplicate lines, or bogus values for PL PH.

=== Details for PL PH ===

(Stub. This is where the correct calculation of PL PH values will go.)

== Format documentation ==

* [https://m100.bitchin100.narkive.com/eP2uSl4J/tokenized-basic-programs-bytes-of-mystery Discussion of byte format on Bitchin 100 Mailing List]
* [http://www.club100.org/library/ups/tokens.do BASIC program to create a table of tokens]

== Other links ==

* [https://github.com/hackerb9/tokenize/ Hackerb9's simple program for tokenizing BASIC code]
* [https://sourceforge.net/projects/virtualt/ A Tandy 200 & Model 100/102 emulator]
* [http://bitchin100.com/CloudT/ A TRS-80 Model 100 emulator that runs in a web browser]

[[Category:Microsoft]]

Tandy 200 BASIC tokenized file

2022-10-05T12:39:56Z

Hackerb9: Add note that VirtualT requires PL PH to be set exactly

Tandy 200 BASIC tokenized file

2022-09-24T05:57:13Z

Hackerb9: Model 100, Tandy 102 appear identical

Tandy 200 BASIC tokenized file

2022-08-21T09:26:07Z

Hackerb9: /* Tokens */ colon is 3A

{{FormatInfo
|subcat=Source code
|subcat2=Tokenized BASIC
|released=1984
}}

'''Tandy 200 BASIC''' was a version of Microsoft BASIC for the Radio Shack Tandy 200 computer. The tokenization for the TRS-80 Model 100/102 and NEC PC-8201/8300 should be similar, if not identical.

== Tokens ==

Blank values indicate either that the token is unused or is used for something unknown.

{| class="wikitable"
! title="Hexadecimal code point" | Hex
! title="Decimal code point" | Dec
! title="BASIC element the token stands for" | Token meaning
! title="Miscellany" | Notes
|-
|-
|80||128||END
|-
|81||129||FOR
|-
|82||130||NEXT
|-
|83||131||DATA
|-
|84||132||INPUT
|-
|85||133||DIM
|-
|86||134||READ
|-
|87||135||LET
|-
|88||136||GOTO
|-
|89||137||RUN
|-
|8A||138||IF
|-
|8B||139||RESTORE
|-
|8C||140||GOSUB
|-
|8D||141||RETURN
|-
|8E||142||REM
|-
|8F||143||STOP
|-
|90||144||WIDTH
|-
|91||145||ELSE||When tokenizing, the Tandy 200 always adds a colon (':') before the ELSE token (3A 91). If the user actually writes :ELSE it is tokenized as 3A 3A 91.
|-
|92||146||LINE
|-
|93||147||EDIT
|-
|94||148||ERROR
|-
|95||149||RESUME
|-
|96||150||OUT
|-
|97||151||ON
|-
|98||152||DSKO$
|-
|99||153||OPEN
|-
|9A||154||CLOSE
|-
|9B||155||LOAD||Note that LOADM is simply the token for LOAD followed by an ASCII 'M'.
|-
|9C||156||MERGE
|-QUOTE
|9D||157||FILES
|-
|9E||158||SAVE
|-
|9F||159||LFILES
|-
|A0||160||LPRINT
|-
|A1||161||DEF
|-
|A2||162||POKE
|-
|A3||163||PRINT
|-
|A4||164||CONT
|-
|A5||165||LIST
|-
|A6||166||LLIST
|-
|A7||167||CLEAR
|-
|A8||168||CLOAD
|-
|A9||169||CSAVE
|-
|AA||170||TIME$
|-
|AB||171||DATE$
|-
|AC||172||DAY$
|-
|AD||173||COM
|-
|AE||174||MDM
|-
|AF||175||KEY
|-
|B0||176||CLS
|-
|B1||177||BEEP
|-
|B2||178||SOUND
|-
|B3||179||LCOPY
|-
|B4||180||PSET
|-
|B5||181||PRESET
|-
|B6||182||MOTOR
|-
|B7||183||MAX
|-
|B8||184||POWER
|-
|B9||185||CALL
|-
|BA||186||MENU
|-
|BB||187||IPL
|-
|BC||188||NAME
|-
|BD||189||KILL
|-
|BE||190||SCREEN
|-
|BF||191||NEW
|-
|C0||192||TAB(
|-
|C1||193||TO
|-
|C2||194||USING
|-
|C3||195||VARPTR
|-
|C4||196||ERL
|-
|C5||197||ERR
|-
|C6||198||STRING$
|-
|C7||199||INSTR
|-
|C8||200||DSKI$
|-
|C9||201||INKEY$
|-
|CA||202||CSRLIN
|-
|CB||203||OFF
|-
|CC||204||HIMEM
|-
|CD||205||THEN
|-
|CE||206||NOT
|-
|CF||207||STEP
|-
|D0||208||+
|-
|D1||209||-
|-
|D2||210||*
|-
|D3||211||/
|-
|D4||212||^
|-
|D5||213||AND
|-
|D6||214||OR
|-
|D7||215||XOR
|-
|D8||216||EQV
|-
|D9||217||IMP
|-
|DA||218||MOD
|-
|DB||219||\
|-
|DC||220||>
|-
|DD||221||=
|-
|DE||222||\|
|-
|DF||223||SGN
|-
|E0||224||INT
|-
|E1||225||ABS
|-
|E2||226||FRE
|-
|E3||227||INP
|-
|E4||228||LPOS
|-
|E5||229||POS
|-
|E6||230||SQR
|-
|E7||231||RND
|-
|E8||232||LOG
|-
|E9||233||EXP
|-
|EA||234||COS
|-
|EB||235||SIN
|-
|EC||236||TAN
|-
|ED||237||ATN
|-
|EE||238||PEEK
|-
|EF||239||EOF
|-
|F0||240||LOC
|-
|F1||241||LOF
|-
|F2||242||CINT
|-
|F3||243||CSNG
|-
|F4||244||CDBL
|-
|F5||245||FIX
|-
|F6||246||LEN
|-
|F7||247||STR$
|-
|F8||248||VAL
|-
|F9||249||ASC
|-
|FA||250||CHR$
|-
|FB||251||SPACE$
|-
|FC||252||LEFT$
|-
|FD||253||RIGHT$
|-
|FE||254||MID$
|-
|FF||255||' (QUOTE) || When tokenizing, the single quote character expands to three characters: a colon (3A), the byte for REM (8E), and then FF.
|}

== File Format ==

Tokenised BASIC code is a sequence of tokenized lines. Each tokenized line has the following format:

{|class="wikitable"
!Name!!Length!!Description
|-
|PL PH
|2 bytes
|Address of the next line in memory, little endian. This data can be ignored as it is only a placeholder. The value of PL PH in a file is moot because the address is recalculated every time the program loads. Once in memory, PL PH is used to skip to the line required for GOTO, GOSUB, and READ, thus speeding up the program.
|-
|LL LH
|2 bytes
|Line number, little endian.
|-
|B0 … BN
|Any number
|Sequence of tokens or ASCII characters. If a character's value is ≥ 128, then it is a token. All tokens are a single byte.
|-
|NULL
|1 byte
|NULL byte (`0x00`) to signal end of line.
|}

== Format documentation ==

* [https://m100.bitchin100.narkive.com/eP2uSl4J/tokenized-basic-programs-bytes-of-mystery Discussion of byte format on Bitchin 100 Mailing List]
* [http://www.club100.org/library/ups/tokens.do BASIC program to create a table of tokens]

== Other links ==

* [https://github.com/hackerb9/tokenize/ Hackerb9's simple program for tokenizing BASIC code]
* [https://sourceforge.net/projects/virtualt/ A Tandy 200 & Model 100/102 emulator]
* [http://bitchin100.com/CloudT/ A TRS-80 Model 100 emulator that runs in a web browser]

[[Category:Microsoft]]

Tandy 200 BASIC tokenized file

2022-08-21T09:24:20Z

Hackerb9: /* Tokens */ clarify :ELSE behaviour

{{FormatInfo
|subcat=Source code
|subcat2=Tokenized BASIC
|released=1984
}}

'''Tandy 200 BASIC''' was a version of Microsoft BASIC for the Radio Shack Tandy 200 computer. The tokenization for the TRS-80 Model 100/102 and NEC PC-8201/8300 should be similar, if not identical.

== Tokens ==

Blank values indicate either that the token is unused or is used for something unknown.

{| class="wikitable"
! title="Hexadecimal code point" | Hex
! title="Decimal code point" | Dec
! title="BASIC element the token stands for" | Token meaning
! title="Miscellany" | Notes
|-
|-
|80||128||END
|-
|81||129||FOR
|-
|82||130||NEXT
|-
|83||131||DATA
|-
|84||132||INPUT
|-
|85||133||DIM
|-
|86||134||READ
|-
|87||135||LET
|-
|88||136||GOTO
|-
|89||137||RUN
|-
|8A||138||IF
|-
|8B||139||RESTORE
|-
|8C||140||GOSUB
|-
|8D||141||RETURN
|-
|8E||142||REM
|-
|8F||143||STOP
|-
|90||144||WIDTH
|-
|91||145||ELSE||When tokenizing, the Tandy 200 always adds a colon (':') before the ELSE token (3A 91). If the user actually writes :ELSE it is tokenized as 3A 3A 91.
|-
|92||146||LINE
|-
|93||147||EDIT
|-
|94||148||ERROR
|-
|95||149||RESUME
|-
|96||150||OUT
|-
|97||151||ON
|-
|98||152||DSKO$
|-
|99||153||OPEN
|-
|9A||154||CLOSE
|-
|9B||155||LOAD||Note that LOADM is simply the token for LOAD followed by an ASCII 'M'.
|-
|9C||156||MERGE
|-QUOTE
|9D||157||FILES
|-
|9E||158||SAVE
|-
|9F||159||LFILES
|-
|A0||160||LPRINT
|-
|A1||161||DEF
|-
|A2||162||POKE
|-
|A3||163||PRINT
|-
|A4||164||CONT
|-
|A5||165||LIST
|-
|A6||166||LLIST
|-
|A7||167||CLEAR
|-
|A8||168||CLOAD
|-
|A9||169||CSAVE
|-
|AA||170||TIME$
|-
|AB||171||DATE$
|-
|AC||172||DAY$
|-
|AD||173||COM
|-
|AE||174||MDM
|-
|AF||175||KEY
|-
|B0||176||CLS
|-
|B1||177||BEEP
|-
|B2||178||SOUND
|-
|B3||179||LCOPY
|-
|B4||180||PSET
|-
|B5||181||PRESET
|-
|B6||182||MOTOR
|-
|B7||183||MAX
|-
|B8||184||POWER
|-
|B9||185||CALL
|-
|BA||186||MENU
|-
|BB||187||IPL
|-
|BC||188||NAME
|-
|BD||189||KILL
|-
|BE||190||SCREEN
|-
|BF||191||NEW
|-
|C0||192||TAB(
|-
|C1||193||TO
|-
|C2||194||USING
|-
|C3||195||VARPTR
|-
|C4||196||ERL
|-
|C5||197||ERR
|-
|C6||198||STRING$
|-
|C7||199||INSTR
|-
|C8||200||DSKI$
|-
|C9||201||INKEY$
|-
|CA||202||CSRLIN
|-
|CB||203||OFF
|-
|CC||204||HIMEM
|-
|CD||205||THEN
|-
|CE||206||NOT
|-
|CF||207||STEP
|-
|D0||208||+
|-
|D1||209||-
|-
|D2||210||*
|-
|D3||211||/
|-
|D4||212||^
|-
|D5||213||AND
|-
|D6||214||OR
|-
|D7||215||XOR
|-
|D8||216||EQV
|-
|D9||217||IMP
|-
|DA||218||MOD
|-
|DB||219||\
|-
|DC||220||>
|-
|DD||221||=
|-
|DE||222||\|
|-
|DF||223||SGN
|-
|E0||224||INT
|-
|E1||225||ABS
|-
|E2||226||FRE
|-
|E3||227||INP
|-
|E4||228||LPOS
|-
|E5||229||POS
|-
|E6||230||SQR
|-
|E7||231||RND
|-
|E8||232||LOG
|-
|E9||233||EXP
|-
|EA||234||COS
|-
|EB||235||SIN
|-
|EC||236||TAN
|-
|ED||237||ATN
|-
|EE||238||PEEK
|-
|EF||239||EOF
|-
|F0||240||LOC
|-
|F1||241||LOF
|-
|F2||242||CINT
|-
|F3||243||CSNG
|-
|F4||244||CDBL
|-
|F5||245||FIX
|-
|F6||246||LEN
|-
|F7||247||STR$
|-
|F8||248||VAL
|-
|F9||249||ASC
|-
|FA||250||CHR$
|-
|FB||251||SPACE$
|-
|FC||252||LEFT$
|-
|FD||253||RIGHT$
|-
|FE||254||MID$
|-
|FF||255||' (QUOTE) || When tokenizing, the single quote character expands to three characters: a colon, the byte for REM (0x8E), and then 0xFF.
|}

== File Format ==

Tokenised BASIC code is a sequence of tokenized lines. Each tokenized line has the following format:

{|class="wikitable"
!Name!!Length!!Description
|-
|PL PH
|2 bytes
|Address of the next line in memory, little endian. This data can be ignored as it is only a placeholder. The value of PL PH in a file is moot because the address is recalculated every time the program loads. Once in memory, PL PH is used to skip to the line required for GOTO, GOSUB, and READ, thus speeding up the program.
|-
|LL LH
|2 bytes
|Line number, little endian.
|-
|B0 … BN
|Any number
|Sequence of tokens or ASCII characters. If a character's value is ≥ 128, then it is a token. All tokens are a single byte.
|-
|NULL
|1 byte
|NULL byte (`0x00`) to signal end of line.
|}

== Format documentation ==

* [https://m100.bitchin100.narkive.com/eP2uSl4J/tokenized-basic-programs-bytes-of-mystery Discussion of byte format on Bitchin 100 Mailing List]
* [http://www.club100.org/library/ups/tokens.do BASIC program to create a table of tokens]

== Other links ==

* [https://github.com/hackerb9/tokenize/ Hackerb9's simple program for tokenizing BASIC code]
* [https://sourceforge.net/projects/virtualt/ A Tandy 200 & Model 100/102 emulator]
* [http://bitchin100.com/CloudT/ A TRS-80 Model 100 emulator that runs in a web browser]

[[Category:Microsoft]]

Tandy 200 BASIC tokenized file

2022-07-26T06:44:06Z

Hackerb9: /* Other links */ Add link to tokenizer

{{FormatInfo
|subcat=Source code
|subcat2=Tokenized BASIC
|released=1984
}}

'''Tandy 200 BASIC''' was a version of Microsoft BASIC for the Radio Shack Tandy 200 computer. The tokenization for the TRS-80 Model 100/102 and NEC PC-8201/8300 should be similar, if not identical.

== Tokens ==

Blank values indicate either that the token is unused or is used for something unknown.

{| class="wikitable"
! title="Hexadecimal code point" | Hex
! title="Decimal code point" | Dec
! title="BASIC element the token stands for" | Token meaning
! title="Miscellany" | Notes
|-
|-
|80||128||END
|-
|81||129||FOR
|-
|82||130||NEXT
|-
|83||131||DATA
|-
|84||132||INPUT
|-
|85||133||DIM
|-
|86||134||READ
|-
|87||135||LET
|-
|88||136||GOTO
|-
|89||137||RUN
|-
|8A||138||IF
|-
|8B||139||RESTORE
|-
|8C||140||GOSUB
|-
|8D||141||RETURN
|-
|8E||142||REM
|-
|8F||143||STOP
|-
|90||144||WIDTH
|-
|91||145||ELSE||When tokenizing, the Tandy 200 always adds a colon (':') before the ELSE token
|-
|92||146||LINE
|-
|93||147||EDIT
|-
|94||148||ERROR
|-
|95||149||RESUME
|-
|96||150||OUT
|-
|97||151||ON
|-
|98||152||DSKO$
|-
|99||153||OPEN
|-
|9A||154||CLOSE
|-
|9B||155||LOAD||Note that LOADM is simply the token for LOAD followed by an ASCII 'M'.
|-
|9C||156||MERGE
|-QUOTE
|9D||157||FILES
|-
|9E||158||SAVE
|-
|9F||159||LFILES
|-
|A0||160||LPRINT
|-
|A1||161||DEF
|-
|A2||162||POKE
|-
|A3||163||PRINT
|-
|A4||164||CONT
|-
|A5||165||LIST
|-
|A6||166||LLIST
|-
|A7||167||CLEAR
|-
|A8||168||CLOAD
|-
|A9||169||CSAVE
|-
|AA||170||TIME$
|-
|AB||171||DATE$
|-
|AC||172||DAY$
|-
|AD||173||COM
|-
|AE||174||MDM
|-
|AF||175||KEY
|-
|B0||176||CLS
|-
|B1||177||BEEP
|-
|B2||178||SOUND
|-
|B3||179||LCOPY
|-
|B4||180||PSET
|-
|B5||181||PRESET
|-
|B6||182||MOTOR
|-
|B7||183||MAX
|-
|B8||184||POWER
|-
|B9||185||CALL
|-
|BA||186||MENU
|-
|BB||187||IPL
|-
|BC||188||NAME
|-
|BD||189||KILL
|-
|BE||190||SCREEN
|-
|BF||191||NEW
|-
|C0||192||TAB(
|-
|C1||193||TO
|-
|C2||194||USING
|-
|C3||195||VARPTR
|-
|C4||196||ERL
|-
|C5||197||ERR
|-
|C6||198||STRING$
|-
|C7||199||INSTR
|-
|C8||200||DSKI$
|-
|C9||201||INKEY$
|-
|CA||202||CSRLIN
|-
|CB||203||OFF
|-
|CC||204||HIMEM
|-
|CD||205||THEN
|-
|CE||206||NOT
|-
|CF||207||STEP
|-
|D0||208||+
|-
|D1||209||-
|-
|D2||210||*
|-
|D3||211||/
|-
|D4||212||^
|-
|D5||213||AND
|-
|D6||214||OR
|-
|D7||215||XOR
|-
|D8||216||EQV
|-
|D9||217||IMP
|-
|DA||218||MOD
|-
|DB||219||\
|-
|DC||220||>
|-
|DD||221||=
|-
|DE||222||\|
|-
|DF||223||SGN
|-
|E0||224||INT
|-
|E1||225||ABS
|-
|E2||226||FRE
|-
|E3||227||INP
|-
|E4||228||LPOS
|-
|E5||229||POS
|-
|E6||230||SQR
|-
|E7||231||RND
|-
|E8||232||LOG
|-
|E9||233||EXP
|-
|EA||234||COS
|-
|EB||235||SIN
|-
|EC||236||TAN
|-
|ED||237||ATN
|-
|EE||238||PEEK
|-
|EF||239||EOF
|-
|F0||240||LOC
|-
|F1||241||LOF
|-
|F2||242||CINT
|-
|F3||243||CSNG
|-
|F4||244||CDBL
|-
|F5||245||FIX
|-
|F6||246||LEN
|-
|F7||247||STR$
|-
|F8||248||VAL
|-
|F9||249||ASC
|-
|FA||250||CHR$
|-
|FB||251||SPACE$
|-
|FC||252||LEFT$
|-
|FD||253||RIGHT$
|-
|FE||254||MID$
|-
|FF||255||' (QUOTE) || When tokenizing, the single quote character expands to three characters: a colon, the byte for REM (0x8E), and then 0xFF.
|}

== File Format ==

Tokenised BASIC code is a sequence of tokenized lines. Each tokenized line has the following format:

{|class="wikitable"
!Name!!Length!!Description
|-
|PL PH
|2 bytes
|Address of the next line in memory, little endian. This data can be ignored as it is only a placeholder. The value of PL PH in a file is moot because the address is recalculated every time the program loads. Once in memory, PL PH is used to skip to the line required for GOTO, GOSUB, and READ, thus speeding up the program.
|-
|LL LH
|2 bytes
|Line number, little endian.
|-
|B0 … BN
|Any number
|Sequence of tokens or ASCII characters. If a character's value is ≥ 128, then it is a token. All tokens are a single byte.
|-
|NULL
|1 byte
|NULL byte (`0x00`) to signal end of line.
|}

== Format documentation ==

* [https://m100.bitchin100.narkive.com/eP2uSl4J/tokenized-basic-programs-bytes-of-mystery Discussion of byte format on Bitchin 100 Mailing List]
* [http://www.club100.org/library/ups/tokens.do BASIC program to create a table of tokens]

== Other links ==

* [https://github.com/hackerb9/tokenize/ Hackerb9's simple program for tokenizing BASIC code]
* [https://sourceforge.net/projects/virtualt/ A Tandy 200 & Model 100/102 emulator]
* [http://bitchin100.com/CloudT/ A TRS-80 Model 100 emulator that runs in a web browser]

[[Category:Microsoft]]

Tandy 200 BASIC tokenized file

2022-07-26T06:38:09Z

Hackerb9: /* Tokens */ Add notes for oddly behaving tokens (ELSE, LOADM, single quote)

{{FormatInfo
|subcat=Source code
|subcat2=Tokenized BASIC
|released=1984
}}

'''Tandy 200 BASIC''' was a version of Microsoft BASIC for the Radio Shack Tandy 200 computer. The tokenization for the TRS-80 Model 100/102 and NEC PC-8201/8300 should be similar, if not identical.

== Tokens ==

Blank values indicate either that the token is unused or is used for something unknown.

{| class="wikitable"
! title="Hexadecimal code point" | Hex
! title="Decimal code point" | Dec
! title="BASIC element the token stands for" | Token meaning
! title="Miscellany" | Notes
|-
|-
|80||128||END
|-
|81||129||FOR
|-
|82||130||NEXT
|-
|83||131||DATA
|-
|84||132||INPUT
|-
|85||133||DIM
|-
|86||134||READ
|-
|87||135||LET
|-
|88||136||GOTO
|-
|89||137||RUN
|-
|8A||138||IF
|-
|8B||139||RESTORE
|-
|8C||140||GOSUB
|-
|8D||141||RETURN
|-
|8E||142||REM
|-
|8F||143||STOP
|-
|90||144||WIDTH
|-
|91||145||ELSE||When tokenizing, the Tandy 200 always adds a colon (':') before the ELSE token
|-
|92||146||LINE
|-
|93||147||EDIT
|-
|94||148||ERROR
|-
|95||149||RESUME
|-
|96||150||OUT
|-
|97||151||ON
|-
|98||152||DSKO$
|-
|99||153||OPEN
|-
|9A||154||CLOSE
|-
|9B||155||LOAD||Note that LOADM is simply the token for LOAD followed by an ASCII 'M'.
|-
|9C||156||MERGE
|-QUOTE
|9D||157||FILES
|-
|9E||158||SAVE
|-
|9F||159||LFILES
|-
|A0||160||LPRINT
|-
|A1||161||DEF
|-
|A2||162||POKE
|-
|A3||163||PRINT
|-
|A4||164||CONT
|-
|A5||165||LIST
|-
|A6||166||LLIST
|-
|A7||167||CLEAR
|-
|A8||168||CLOAD
|-
|A9||169||CSAVE
|-
|AA||170||TIME$
|-
|AB||171||DATE$
|-
|AC||172||DAY$
|-
|AD||173||COM
|-
|AE||174||MDM
|-
|AF||175||KEY
|-
|B0||176||CLS
|-
|B1||177||BEEP
|-
|B2||178||SOUND
|-
|B3||179||LCOPY
|-
|B4||180||PSET
|-
|B5||181||PRESET
|-
|B6||182||MOTOR
|-
|B7||183||MAX
|-
|B8||184||POWER
|-
|B9||185||CALL
|-
|BA||186||MENU
|-
|BB||187||IPL
|-
|BC||188||NAME
|-
|BD||189||KILL
|-
|BE||190||SCREEN
|-
|BF||191||NEW
|-
|C0||192||TAB(
|-
|C1||193||TO
|-
|C2||194||USING
|-
|C3||195||VARPTR
|-
|C4||196||ERL
|-
|C5||197||ERR
|-
|C6||198||STRING$
|-
|C7||199||INSTR
|-
|C8||200||DSKI$
|-
|C9||201||INKEY$
|-
|CA||202||CSRLIN
|-
|CB||203||OFF
|-
|CC||204||HIMEM
|-
|CD||205||THEN
|-
|CE||206||NOT
|-
|CF||207||STEP
|-
|D0||208||+
|-
|D1||209||-
|-
|D2||210||*
|-
|D3||211||/
|-
|D4||212||^
|-
|D5||213||AND
|-
|D6||214||OR
|-
|D7||215||XOR
|-
|D8||216||EQV
|-
|D9||217||IMP
|-
|DA||218||MOD
|-
|DB||219||\
|-
|DC||220||>
|-
|DD||221||=
|-
|DE||222||\|
|-
|DF||223||SGN
|-
|E0||224||INT
|-
|E1||225||ABS
|-
|E2||226||FRE
|-
|E3||227||INP
|-
|E4||228||LPOS
|-
|E5||229||POS
|-
|E6||230||SQR
|-
|E7||231||RND
|-
|E8||232||LOG
|-
|E9||233||EXP
|-
|EA||234||COS
|-
|EB||235||SIN
|-
|EC||236||TAN
|-
|ED||237||ATN
|-
|EE||238||PEEK
|-
|EF||239||EOF
|-
|F0||240||LOC
|-
|F1||241||LOF
|-
|F2||242||CINT
|-
|F3||243||CSNG
|-
|F4||244||CDBL
|-
|F5||245||FIX
|-
|F6||246||LEN
|-
|F7||247||STR$
|-
|F8||248||VAL
|-
|F9||249||ASC
|-
|FA||250||CHR$
|-
|FB||251||SPACE$
|-
|FC||252||LEFT$
|-
|FD||253||RIGHT$
|-
|FE||254||MID$
|-
|FF||255||' (QUOTE) || When tokenizing, the single quote character expands to three characters: a colon, the byte for REM (0x8E), and then 0xFF.
|}

== File Format ==

Tokenised BASIC code is a sequence of tokenized lines. Each tokenized line has the following format:

{|class="wikitable"
!Name!!Length!!Description
|-
|PL PH
|2 bytes
|Address of the next line in memory, little endian. This data can be ignored as it is only a placeholder. The value of PL PH in a file is moot because the address is recalculated every time the program loads. Once in memory, PL PH is used to skip to the line required for GOTO, GOSUB, and READ, thus speeding up the program.
|-
|LL LH
|2 bytes
|Line number, little endian.
|-
|B0 … BN
|Any number
|Sequence of tokens or ASCII characters. If a character's value is ≥ 128, then it is a token. All tokens are a single byte.
|-
|NULL
|1 byte
|NULL byte (`0x00`) to signal end of line.
|}

== Format documentation ==

* [https://m100.bitchin100.narkive.com/eP2uSl4J/tokenized-basic-programs-bytes-of-mystery Discussion of byte format on Bitchin 100 Mailing List]
* [http://www.club100.org/library/ups/tokens.do BASIC program to create a table of tokens]

== Other links ==

* [https://sourceforge.net/projects/virtualt/ A Tandy 200 & Model 100/102 emulator]
* [http://bitchin100.com/CloudT/ A TRS-80 Model 100 emulator that runs in a web browser]

[[Category:Microsoft]]

Tandy 200 BASIC tokenized file

2022-07-26T06:29:05Z

Hackerb9: /* File Format */ File does not end line number set to 0x00 0x00.

{{FormatInfo
|subcat=Source code
|subcat2=Tokenized BASIC
|released=1984
}}

'''Tandy 200 BASIC''' was a version of Microsoft BASIC for the Radio Shack Tandy 200 computer. The tokenization for the TRS-80 Model 100/102 and NEC PC-8201/8300 should be similar, if not identical.

== Tokens ==

Blank values indicate either that the token is unused or is used for something unknown.

{| class="wikitable"
! title="Hexadecimal code point" | Hex
! title="Decimal code point" | Dec
! title="BASIC element the token stands for" | Token meaning
|-
|-
|80||128||END
|-
|81||129||FOR
|-
|82||130||NEXT
|-
|83||131||DATA
|-
|84||132||INPUT
|-
|85||133||DIM
|-
|86||134||READ
|-
|87||135||LET
|-
|88||136||GOTO
|-
|89||137||RUN
|-
|8A||138||IF
|-
|8B||139||RESTORE
|-
|8C||140||GOSUB
|-
|8D||141||RETURN
|-
|8E||142||REM
|-
|8F||143||STOP
|-
|90||144||WIDTH
|-
|91||145||ELSE
|-
|92||146||LINE
|-
|93||147||EDIT
|-
|94||148||ERROR
|-
|95||149||RESUME
|-
|96||150||OUT
|-
|97||151||ON
|-
|98||152||DSKO$
|-
|99||153||OPEN
|-
|9A||154||CLOSE
|-
|9B||155||LOAD
|-
|9C||156||MERGE
|-
|9D||157||FILES
|-
|9E||158||SAVE
|-
|9F||159||LFILES
|-
|A0||160||LPRINT
|-
|A1||161||DEF
|-
|A2||162||POKE
|-
|A3||163||PRINT
|-
|A4||164||CONT
|-
|A5||165||LIST
|-
|A6||166||LLIST
|-
|A7||167||CLEAR
|-
|A8||168||CLOAD
|-
|A9||169||CSAVE
|-
|AA||170||TIME$
|-
|AB||171||DATE$
|-
|AC||172||DAY$
|-
|AD||173||COM
|-
|AE||174||MDM
|-
|AF||175||KEY
|-
|B0||176||CLS
|-
|B1||177||BEEP
|-
|B2||178||SOUND
|-
|B3||179||LCOPY
|-
|B4||180||PSET
|-
|B5||181||PRESET
|-
|B6||182||MOTOR
|-
|B7||183||MAX
|-
|B8||184||POWER
|-
|B9||185||CALL
|-
|BA||186||MENU
|-
|BB||187||IPL
|-
|BC||188||NAME
|-
|BD||189||KILL
|-
|BE||190||SCREEN
|-
|BF||191||NEW
|-
|C0||192||TAB(
|-
|C1||193||TO
|-
|C2||194||USING
|-
|C3||195||VARPTR
|-
|C4||196||ERL
|-
|C5||197||ERR
|-
|C6||198||STRING$
|-
|C7||199||INSTR
|-
|C8||200||DSKI$
|-
|C9||201||INKEY$
|-
|CA||202||CSRLIN
|-
|CB||203||OFF
|-
|CC||204||HIMEM
|-
|CD||205||THEN
|-
|CE||206||NOT
|-
|CF||207||STEP
|-
|D0||208||+
|-
|D1||209||-
|-
|D2||210||*
|-
|D3||211||/
|-
|D4||212||^
|-
|D5||213||AND
|-
|D6||214||OR
|-
|D7||215||XOR
|-
|D8||216||EQV
|-
|D9||217||IMP
|-
|DA||218||MOD
|-
|DB||219||\
|-
|DC||220||>
|-
|DD||221||=
|-
|DE||222||\|
|-
|DF||223||SGN
|-
|E0||224||INT
|-
|E1||225||ABS
|-
|E2||226||FRE
|-
|E3||227||INP
|-
|E4||228||LPOS
|-
|E5||229||POS
|-
|E6||230||SQR
|-
|E7||231||RND
|-
|E8||232||LOG
|-
|E9||233||EXP
|-
|EA||234||COS
|-
|EB||235||SIN
|-
|EC||236||TAN
|-
|ED||237||ATN
|-
|EE||238||PEEK
|-
|EF||239||EOF
|-
|F0||240||LOC
|-
|F1||241||LOF
|-
|F2||242||CINT
|-
|F3||243||CSNG
|-
|F4||244||CDBL
|-
|F5||245||FIX
|-
|F6||246||LEN
|-
|F7||247||STR$
|-
|F8||248||VAL
|-
|F9||249||ASC
|-
|FA||250||CHR$
|-
|FB||251||SPACE$
|-
|FC||252||LEFT$
|-
|FD||253||RIGHT$
|-
|FE||254||MID$
|-
|FF||255||' (QUOTE)
|}

== File Format ==

Tokenised BASIC code is a sequence of tokenized lines. Each tokenized line has the following format:

{|class="wikitable"
!Name!!Length!!Description
|-
|PL PH
|2 bytes
|Address of the next line in memory, little endian. This data can be ignored as it is only a placeholder. The value of PL PH in a file is moot because the address is recalculated every time the program loads. Once in memory, PL PH is used to skip to the line required for GOTO, GOSUB, and READ, thus speeding up the program.
|-
|LL LH
|2 bytes
|Line number, little endian.
|-
|B0 … BN
|Any number
|Sequence of tokens or ASCII characters. If a character's value is ≥ 128, then it is a token. All tokens are a single byte.
|-
|NULL
|1 byte
|NULL byte (`0x00`) to signal end of line.
|}

== Format documentation ==

* [https://m100.bitchin100.narkive.com/eP2uSl4J/tokenized-basic-programs-bytes-of-mystery Discussion of byte format on Bitchin 100 Mailing List]
* [http://www.club100.org/library/ups/tokens.do BASIC program to create a table of tokens]

== Other links ==

* [https://sourceforge.net/projects/virtualt/ A Tandy 200 & Model 100/102 emulator]
* [http://bitchin100.com/CloudT/ A TRS-80 Model 100 emulator that runs in a web browser]

[[Category:Microsoft]]

Tandy 200 BASIC tokenized file

2022-07-25T06:05:56Z

Hackerb9: /* File Format */ Add specific details on the bytes in the file.

{{FormatInfo
|subcat=Source code
|subcat2=Tokenized BASIC
|released=1984
}}

'''Tandy 200 BASIC''' was a version of Microsoft BASIC for the Radio Shack Tandy 200 computer. The tokenization for the TRS-80 Model 100/102 and NEC PC-8201/8300 should be similar, if not identical.

== Tokens ==

Blank values indicate either that the token is unused or is used for something unknown.

{| class="wikitable"
! title="Hexadecimal code point" | Hex
! title="Decimal code point" | Dec
! title="BASIC element the token stands for" | Token meaning
|-
|-
|80||128||END
|-
|81||129||FOR
|-
|82||130||NEXT
|-
|83||131||DATA
|-
|84||132||INPUT
|-
|85||133||DIM
|-
|86||134||READ
|-
|87||135||LET
|-
|88||136||GOTO
|-
|89||137||RUN
|-
|8A||138||IF
|-
|8B||139||RESTORE
|-
|8C||140||GOSUB
|-
|8D||141||RETURN
|-
|8E||142||REM
|-
|8F||143||STOP
|-
|90||144||WIDTH
|-
|91||145||ELSE
|-
|92||146||LINE
|-
|93||147||EDIT
|-
|94||148||ERROR
|-
|95||149||RESUME
|-
|96||150||OUT
|-
|97||151||ON
|-
|98||152||DSKO$
|-
|99||153||OPEN
|-
|9A||154||CLOSE
|-
|9B||155||LOAD
|-
|9C||156||MERGE
|-
|9D||157||FILES
|-
|9E||158||SAVE
|-
|9F||159||LFILES
|-
|A0||160||LPRINT
|-
|A1||161||DEF
|-
|A2||162||POKE
|-
|A3||163||PRINT
|-
|A4||164||CONT
|-
|A5||165||LIST
|-
|A6||166||LLIST
|-
|A7||167||CLEAR
|-
|A8||168||CLOAD
|-
|A9||169||CSAVE
|-
|AA||170||TIME$
|-
|AB||171||DATE$
|-
|AC||172||DAY$
|-
|AD||173||COM
|-
|AE||174||MDM
|-
|AF||175||KEY
|-
|B0||176||CLS
|-
|B1||177||BEEP
|-
|B2||178||SOUND
|-
|B3||179||LCOPY
|-
|B4||180||PSET
|-
|B5||181||PRESET
|-
|B6||182||MOTOR
|-
|B7||183||MAX
|-
|B8||184||POWER
|-
|B9||185||CALL
|-
|BA||186||MENU
|-
|BB||187||IPL
|-
|BC||188||NAME
|-
|BD||189||KILL
|-
|BE||190||SCREEN
|-
|BF||191||NEW
|-
|C0||192||TAB(
|-
|C1||193||TO
|-
|C2||194||USING
|-
|C3||195||VARPTR
|-
|C4||196||ERL
|-
|C5||197||ERR
|-
|C6||198||STRING$
|-
|C7||199||INSTR
|-
|C8||200||DSKI$
|-
|C9||201||INKEY$
|-
|CA||202||CSRLIN
|-
|CB||203||OFF
|-
|CC||204||HIMEM
|-
|CD||205||THEN
|-
|CE||206||NOT
|-
|CF||207||STEP
|-
|D0||208||+
|-
|D1||209||-
|-
|D2||210||*
|-
|D3||211||/
|-
|D4||212||^
|-
|D5||213||AND
|-
|D6||214||OR
|-
|D7||215||XOR
|-
|D8||216||EQV
|-
|D9||217||IMP
|-
|DA||218||MOD
|-
|DB||219||\
|-
|DC||220||>
|-
|DD||221||=
|-
|DE||222||\|
|-
|DF||223||SGN
|-
|E0||224||INT
|-
|E1||225||ABS
|-
|E2||226||FRE
|-
|E3||227||INP
|-
|E4||228||LPOS
|-
|E5||229||POS
|-
|E6||230||SQR
|-
|E7||231||RND
|-
|E8||232||LOG
|-
|E9||233||EXP
|-
|EA||234||COS
|-
|EB||235||SIN
|-
|EC||236||TAN
|-
|ED||237||ATN
|-
|EE||238||PEEK
|-
|EF||239||EOF
|-
|F0||240||LOC
|-
|F1||241||LOF
|-
|F2||242||CINT
|-
|F3||243||CSNG
|-
|F4||244||CDBL
|-
|F5||245||FIX
|-
|F6||246||LEN
|-
|F7||247||STR$
|-
|F8||248||VAL
|-
|F9||249||ASC
|-
|FA||250||CHR$
|-
|FB||251||SPACE$
|-
|FC||252||LEFT$
|-
|FD||253||RIGHT$
|-
|FE||254||MID$
|-
|FF||255||' (QUOTE)
|}

== File Format ==

Tokenised BASIC code is a sequence of tokenized lines. Each tokenized line has the following format:

{|class="wikitable"
!Name!!Length!!Description
|-
|PL PH
|2 bytes
|Address of the next line in memory, little endian. With one exception, this data can be ignored as it is only a placeholder. The exception is that the end of the program listing is marked by PL PH == 00 00. The value of PL PH in a file does not matter because the address is recalculated every time the program loads. PL PH is used to skip to the line required for GOTO, GOSUB, and READ, thus speeding up the program.
|-
|LL LH
|2 bytes
|Line number, little endian.
|-
|B0 … BN
|Any number
|Sequence of tokens or ASCII characters. If a character's value is ≥ 128, then it is a token. All tokens are a single byte.
|-
|NULL
|1 byte
|NULL byte (00) to signal end of line.
|}

== Format documentation ==

* [https://m100.bitchin100.narkive.com/eP2uSl4J/tokenized-basic-programs-bytes-of-mystery Discussion of byte format on Bitchin 100 Mailing List]
* [http://www.club100.org/library/ups/tokens.do BASIC program to create a table of tokens]

== Other links ==

* [https://sourceforge.net/projects/virtualt/ A Tandy 200 & Model 100/102 emulator]
* [http://bitchin100.com/CloudT/ A TRS-80 Model 100 emulator that runs in a web browser]

[[Category:Microsoft]]

Tandy 200 BASIC tokenized file

2022-07-24T11:42:59Z

Hackerb9: /* Tokens */ Apparently REM and ' have separate entries.

{{FormatInfo
|subcat=Source code
|subcat2=Tokenized BASIC
|released=1984
}}

'''Tandy 200 BASIC''' was a version of Microsoft BASIC for the Radio Shack Tandy 200 computer. The tokenization for the TRS-80 Model 100/102 and NEC PC-8201/8300 should be similar, if not identical.

== Tokens ==

Blank values indicate either that the token is unused or is used for something unknown.

{| class="wikitable"
! title="Hexadecimal code point" | Hex
! title="Decimal code point" | Dec
! title="BASIC element the token stands for" | Token meaning
|-
|-
|80||128||END
|-
|81||129||FOR
|-
|82||130||NEXT
|-
|83||131||DATA
|-
|84||132||INPUT
|-
|85||133||DIM
|-
|86||134||READ
|-
|87||135||LET
|-
|88||136||GOTO
|-
|89||137||RUN
|-
|8A||138||IF
|-
|8B||139||RESTORE
|-
|8C||140||GOSUB
|-
|8D||141||RETURN
|-
|8E||142||REM
|-
|8F||143||STOP
|-
|90||144||WIDTH
|-
|91||145||ELSE
|-
|92||146||LINE
|-
|93||147||EDIT
|-
|94||148||ERROR
|-
|95||149||RESUME
|-
|96||150||OUT
|-
|97||151||ON
|-
|98||152||DSKO$
|-
|99||153||OPEN
|-
|9A||154||CLOSE
|-
|9B||155||LOAD
|-
|9C||156||MERGE
|-
|9D||157||FILES
|-
|9E||158||SAVE
|-
|9F||159||LFILES
|-
|A0||160||LPRINT
|-
|A1||161||DEF
|-
|A2||162||POKE
|-
|A3||163||PRINT
|-
|A4||164||CONT
|-
|A5||165||LIST
|-
|A6||166||LLIST
|-
|A7||167||CLEAR
|-
|A8||168||CLOAD
|-
|A9||169||CSAVE
|-
|AA||170||TIME$
|-
|AB||171||DATE$
|-
|AC||172||DAY$
|-
|AD||173||COM
|-
|AE||174||MDM
|-
|AF||175||KEY
|-
|B0||176||CLS
|-
|B1||177||BEEP
|-
|B2||178||SOUND
|-
|B3||179||LCOPY
|-
|B4||180||PSET
|-
|B5||181||PRESET
|-
|B6||182||MOTOR
|-
|B7||183||MAX
|-
|B8||184||POWER
|-
|B9||185||CALL
|-
|BA||186||MENU
|-
|BB||187||IPL
|-
|BC||188||NAME
|-
|BD||189||KILL
|-
|BE||190||SCREEN
|-
|BF||191||NEW
|-
|C0||192||TAB(
|-
|C1||193||TO
|-
|C2||194||USING
|-
|C3||195||VARPTR
|-
|C4||196||ERL
|-
|C5||197||ERR
|-
|C6||198||STRING$
|-
|C7||199||INSTR
|-
|C8||200||DSKI$
|-
|C9||201||INKEY$
|-
|CA||202||CSRLIN
|-
|CB||203||OFF
|-
|CC||204||HIMEM
|-
|CD||205||THEN
|-
|CE||206||NOT
|-
|CF||207||STEP
|-
|D0||208||+
|-
|D1||209||-
|-
|D2||210||*
|-
|D3||211||/
|-
|D4||212||^
|-
|D5||213||AND
|-
|D6||214||OR
|-
|D7||215||XOR
|-
|D8||216||EQV
|-
|D9||217||IMP
|-
|DA||218||MOD
|-
|DB||219||\
|-
|DC||220||>
|-
|DD||221||=
|-
|DE||222||\|
|-
|DF||223||SGN
|-
|E0||224||INT
|-
|E1||225||ABS
|-
|E2||226||FRE
|-
|E3||227||INP
|-
|E4||228||LPOS
|-
|E5||229||POS
|-
|E6||230||SQR
|-
|E7||231||RND
|-
|E8||232||LOG
|-
|E9||233||EXP
|-
|EA||234||COS
|-
|EB||235||SIN
|-
|EC||236||TAN
|-
|ED||237||ATN
|-
|EE||238||PEEK
|-
|EF||239||EOF
|-
|F0||240||LOC
|-
|F1||241||LOF
|-
|F2||242||CINT
|-
|F3||243||CSNG
|-
|F4||244||CDBL
|-
|F5||245||FIX
|-
|F6||246||LEN
|-
|F7||247||STR$
|-
|F8||248||VAL
|-
|F9||249||ASC
|-
|FA||250||CHR$
|-
|FB||251||SPACE$
|-
|FC||252||LEFT$
|-
|FD||253||RIGHT$
|-
|FE||254||MID$
|-
|FF||255||' (QUOTE)
|}

== Format documentation ==
* [http://www.club100.org/library/ups/tokens.do BASIC program to create a table of tokens]

== Other links ==

* [https://sourceforge.net/projects/virtualt/ A Tandy 200 & Model 100/102 emulator]
* [http://bitchin100.com/CloudT/ A TRS-80 Model 100 emulator that runs in a web browser]

[[Category:Microsoft]]

Tandy 200 BASIC tokenized file

2022-07-24T11:41:12Z

Hackerb9: /* Tokens */ Typo: didn't need to escape backslash

{{FormatInfo
|subcat=Source code
|subcat2=Tokenized BASIC
|released=1984
}}

'''Tandy 200 BASIC''' was a version of Microsoft BASIC for the Radio Shack Tandy 200 computer. The tokenization for the TRS-80 Model 100/102 and NEC PC-8201/8300 should be similar, if not identical.

== Tokens ==

Blank values indicate either that the token is unused or is used for something unknown.

{| class="wikitable"
! title="Hexadecimal code point" | Hex
! title="Decimal code point" | Dec
! title="BASIC element the token stands for" | Token meaning
|-
|-
|80||128||END
|-
|81||129||FOR
|-
|82||130||NEXT
|-
|83||131||DATA
|-
|84||132||INPUT
|-
|85||133||DIM
|-
|86||134||READ
|-
|87||135||LET
|-
|88||136||GOTO
|-
|89||137||RUN
|-
|8A||138||IF
|-
|8B||139||RESTORE
|-
|8C||140||GOSUB
|-
|8D||141||RETURN
|-
|8E||142||REM
|-
|8F||143||STOP
|-
|90||144||WIDTH
|-
|91||145||ELSE
|-
|92||146||LINE
|-
|93||147||EDIT
|-
|94||148||ERROR
|-
|95||149||RESUME
|-
|96||150||OUT
|-
|97||151||ON
|-
|98||152||DSKO$
|-
|99||153||OPEN
|-
|9A||154||CLOSE
|-
|9B||155||LOAD
|-
|9C||156||MERGE
|-
|9D||157||FILES
|-
|9E||158||SAVE
|-
|9F||159||LFILES
|-
|A0||160||LPRINT
|-
|A1||161||DEF
|-
|A2||162||POKE
|-
|A3||163||PRINT
|-
|A4||164||CONT
|-
|A5||165||LIST
|-
|A6||166||LLIST
|-
|A7||167||CLEAR
|-
|A8||168||CLOAD
|-
|A9||169||CSAVE
|-
|AA||170||TIME$
|-
|AB||171||DATE$
|-
|AC||172||DAY$
|-
|AD||173||COM
|-
|AE||174||MDM
|-
|AF||175||KEY
|-
|B0||176||CLS
|-
|B1||177||BEEP
|-
|B2||178||SOUND
|-
|B3||179||LCOPY
|-
|B4||180||PSET
|-
|B5||181||PRESET
|-
|B6||182||MOTOR
|-
|B7||183||MAX
|-
|B8||184||POWER
|-
|B9||185||CALL
|-
|BA||186||MENU
|-
|BB||187||IPL
|-
|BC||188||NAME
|-
|BD||189||KILL
|-
|BE||190||SCREEN
|-
|BF||191||NEW
|-
|C0||192||TAB(
|-
|C1||193||TO
|-
|C2||194||USING
|-
|C3||195||VARPTR
|-
|C4||196||ERL
|-
|C5||197||ERR
|-
|C6||198||STRING$
|-
|C7||199||INSTR
|-
|C8||200||DSKI$
|-
|C9||201||INKEY$
|-
|CA||202||CSRLIN
|-
|CB||203||OFF
|-
|CC||204||HIMEM
|-
|CD||205||THEN
|-
|CE||206||NOT
|-
|CF||207||STEP
|-
|D0||208||+
|-
|D1||209||-
|-
|D2||210||*
|-
|D3||211||/
|-
|D4||212||^
|-
|D5||213||AND
|-
|D6||214||OR
|-
|D7||215||XOR
|-
|D8||216||EQV
|-
|D9||217||IMP
|-
|DA||218||MOD
|-
|DB||219||\
|-
|DC||220||>
|-
|DD||221||=
|-
|DE||222||\|
|-
|DF||223||SGN
|-
|E0||224||INT
|-
|E1||225||ABS
|-
|E2||226||FRE
|-
|E3||227||INP
|-
|E4||228||LPOS
|-
|E5||229||POS
|-
|E6||230||SQR
|-
|E7||231||RND
|-
|E8||232||LOG
|-
|E9||233||EXP
|-
|EA||234||COS
|-
|EB||235||SIN
|-
|EC||236||TAN
|-
|ED||237||ATN
|-
|EE||238||PEEK
|-
|EF||239||EOF
|-
|F0||240||LOC
|-
|F1||241||LOF
|-
|F2||242||CINT
|-
|F3||243||CSNG
|-
|F4||244||CDBL
|-
|F5||245||FIX
|-
|F6||246||LEN
|-
|F7||247||STR$
|-
|F8||248||VAL
|-
|F9||249||ASC
|-
|FA||250||CHR$
|-
|FB||251||SPACE$
|-
|FC||252||LEFT$
|-
|FD||253||RIGHT$
|-
|FE||254||MID$
|-
|FF||255||(REM QUOTE)
|}

== Format documentation ==
* [http://www.club100.org/library/ups/tokens.do BASIC program to create a table of tokens]

== Other links ==

* [https://sourceforge.net/projects/virtualt/ A Tandy 200 & Model 100/102 emulator]
* [http://bitchin100.com/CloudT/ A TRS-80 Model 100 emulator that runs in a web browser]

[[Category:Microsoft]]

Tandy 200 BASIC tokenized file

2022-07-24T11:40:25Z

Hackerb9: Created page and added the table of tokens from a genuine Tandy 200

{{FormatInfo
|subcat=Source code
|subcat2=Tokenized BASIC
|released=1984
}}

'''Tandy 200 BASIC''' was a version of Microsoft BASIC for the Radio Shack Tandy 200 computer. The tokenization for the TRS-80 Model 100/102 and NEC PC-8201/8300 should be similar, if not identical.

== Tokens ==

Blank values indicate either that the token is unused or is used for something unknown.

{| class="wikitable"
! title="Hexadecimal code point" | Hex
! title="Decimal code point" | Dec
! title="BASIC element the token stands for" | Token meaning
|-
|-
|80||128||END
|-
|81||129||FOR
|-
|82||130||NEXT
|-
|83||131||DATA
|-
|84||132||INPUT
|-
|85||133||DIM
|-
|86||134||READ
|-
|87||135||LET
|-
|88||136||GOTO
|-
|89||137||RUN
|-
|8A||138||IF
|-
|8B||139||RESTORE
|-
|8C||140||GOSUB
|-
|8D||141||RETURN
|-
|8E||142||REM
|-
|8F||143||STOP
|-
|90||144||WIDTH
|-
|91||145||ELSE
|-
|92||146||LINE
|-
|93||147||EDIT
|-
|94||148||ERROR
|-
|95||149||RESUME
|-
|96||150||OUT
|-
|97||151||ON
|-
|98||152||DSKO$
|-
|99||153||OPEN
|-
|9A||154||CLOSE
|-
|9B||155||LOAD
|-
|9C||156||MERGE
|-
|9D||157||FILES
|-
|9E||158||SAVE
|-
|9F||159||LFILES
|-
|A0||160||LPRINT
|-
|A1||161||DEF
|-
|A2||162||POKE
|-
|A3||163||PRINT
|-
|A4||164||CONT
|-
|A5||165||LIST
|-
|A6||166||LLIST
|-
|A7||167||CLEAR
|-
|A8||168||CLOAD
|-
|A9||169||CSAVE
|-
|AA||170||TIME$
|-
|AB||171||DATE$
|-
|AC||172||DAY$
|-
|AD||173||COM
|-
|AE||174||MDM
|-
|AF||175||KEY
|-
|B0||176||CLS
|-
|B1||177||BEEP
|-
|B2||178||SOUND
|-
|B3||179||LCOPY
|-
|B4||180||PSET
|-
|B5||181||PRESET
|-
|B6||182||MOTOR
|-
|B7||183||MAX
|-
|B8||184||POWER
|-
|B9||185||CALL
|-
|BA||186||MENU
|-
|BB||187||IPL
|-
|BC||188||NAME
|-
|BD||189||KILL
|-
|BE||190||SCREEN
|-
|BF||191||NEW
|-
|C0||192||TAB(
|-
|C1||193||TO
|-
|C2||194||USING
|-
|C3||195||VARPTR
|-
|C4||196||ERL
|-
|C5||197||ERR
|-
|C6||198||STRING$
|-
|C7||199||INSTR
|-
|C8||200||DSKI$
|-
|C9||201||INKEY$
|-
|CA||202||CSRLIN
|-
|CB||203||OFF
|-
|CC||204||HIMEM
|-
|CD||205||THEN
|-
|CE||206||NOT
|-
|CF||207||STEP
|-
|D0||208||+
|-
|D1||209||-
|-
|D2||210||*
|-
|D3||211||/
|-
|D4||212||^
|-
|D5||213||AND
|-
|D6||214||OR
|-
|D7||215||XOR
|-
|D8||216||EQV
|-
|D9||217||IMP
|-
|DA||218||MOD
|-
|DB||219||\\
|-
|DC||220||>
|-
|DD||221||=
|-
|DE||222||\|
|-
|DF||223||SGN
|-
|E0||224||INT
|-
|E1||225||ABS
|-
|E2||226||FRE
|-
|E3||227||INP
|-
|E4||228||LPOS
|-
|E5||229||POS
|-
|E6||230||SQR
|-
|E7||231||RND
|-
|E8||232||LOG
|-
|E9||233||EXP
|-
|EA||234||COS
|-
|EB||235||SIN
|-
|EC||236||TAN
|-
|ED||237||ATN
|-
|EE||238||PEEK
|-
|EF||239||EOF
|-
|F0||240||LOC
|-
|F1||241||LOF
|-
|F2||242||CINT
|-
|F3||243||CSNG
|-
|F4||244||CDBL
|-
|F5||245||FIX
|-
|F6||246||LEN
|-
|F7||247||STR$
|-
|F8||248||VAL
|-
|F9||249||ASC
|-
|FA||250||CHR$
|-
|FB||251||SPACE$
|-
|FC||252||LEFT$
|-
|FD||253||RIGHT$
|-
|FE||254||MID$
|-
|FF||255||(REM QUOTE)
|}

== Format documentation ==
* [http://www.club100.org/library/ups/tokens.do BASIC program to create a table of tokens]

== Other links ==

* [https://sourceforge.net/projects/virtualt/ A Tandy 200 & Model 100/102 emulator]
* [http://bitchin100.com/CloudT/ A TRS-80 Model 100 emulator that runs in a web browser]

[[Category:Microsoft]]