Tandy 200 BASIC tokenized file

Revision as of 15:18, 28 September 2023

Tandy 200 BASIC was a version of Microsoft BASIC for the Radio Shack Tandy 200 computer. The tokenizations for the TRS-80 Model 100 and Tandy 102 appear to be identical. In fact, while the research that created this page was done on a Tandy 200, experiments using emulated systems indicate that the tokenized BASIC file format is the same across most of the computers related to the Kyocera Kyotronic-85, with the exception being the models made by NEC.

While the NEC PC-8201/8300 format is not the same, the BASIC it runs (N82 BASIC) is very close and it is expected the tokenization will be similar.

Tokens

All BASIC tokens on the Tandy 200 are a single byte with a value from 128 to 255.

File Format

Tokenised BASIC code is a sequence of tokenized lines. Each tokenized line has the following format:

Typically, the list of tokenized lines continues until the end of the file. Once a program is loaded into RAM, the end is signaled by two consecutive NULLs in the location where PL PH would normally start the next line.

Limits

• Line numbers can range from 0 to 65529 (but see variances below).
• Tokenized line length is limited to 256 characters, including the trailing NULL byte. (To do: double check this.)

Acceptable variances

While Tandy BASIC will never generate such files, it has no problem loading a tokenized program that:

• Has line numbers out of order; the numerical value of a line number is ignored. Program lines are always LISTed and RUN in the order they appear in the tokenized file.
• Contains duplicate line numbers; all lines are kept and run in the order found in the file. (GOTO is complicated as search starts at the next line, not the beginning of the program.)
• Contains a line with zero bytes of data; LIST will show the line number with no text afterward. Does not replace or delete a preexisting line with that number.
• Contains arbitrary bytes for PL PH; the value of PL PH is always recalculated at LOAD time.
• Contains line numbers greater than the nominal maximum of 65529; this can be used to store hidden binary data according to John R. Hogerhuis.

Sidenote: while these variances are accepted by actual hardware, emulators may not be able to handle such files. For example, Virtual T (as of version 1.7 in 2022) refuses to load tokenized BASIC programs that have lines out of order, duplicate lines, or bogus values for PL PH.

Details for PL PH

In a file, the specific values of the 16-bit integer PL PH are never used. During program load, Tandy BASIC recalculates PL PH before storing it into RAM. Once in memory, PL PH is used to skip to the line required for GOTO, GOSUB, and READ, thus speeding up the program. Additionally, the end of a tokenized BASIC listing in RAM is marked by PL PH equal to zero (two NULL bytes).

(Stub. This is where Tandy BASIC's calculation of PL PH values will go.)

Format documentation

• Discussion of byte format on Bitchin 100 Mailing List
• BASIC program to create a table of tokens
• Hidden Powers of the TRS 80 Model 100. Appendix B lists the BASIC token values for the Model 100. Other than the omission of 255 for a single quote remark, this reference matches what hackerb9 found programmatically for the Tandy 200.
• Ayra Model 100 BASIC Reference

Other links

• Hackerb9's C program for tokenizing BASIC code
• A Tandy 200 & Model 100/102 emulator
• A TRS-80 Model 100 emulator that runs in a web browser