Trace: » forth03.html

[hemmerling] FORTH 3/7

Integrated Development Environments ( IDEs ) & Terminal Software with FORTH Support

The FORTH Development Process

Classical - 100% of static Code Generation on the Host ( Tethered FORTH )

  • The FORTH system boots on the target.
  • Typical in the era of software updates on the target, by buring new EPROMs ( 1975-2000 ).
  • All modern software tools for editing, code generation and source code saving can be used. But at that time, 1975-2000, typically no CVS. No continous delivery though.
  • Waterfall software development process:
    1. Create source code on the host, using a simple editor or an IDE.
    2. Generate code on the host, using an IDE, compiler, assembler.
    3. Optional testing on a hardware & software simulator, on the host.
    4. Burn EPROMs for the target, or flash the complete FlashMemory of the target.
    5. The FORTH application might be executed from FlashMemory or from RAM, this depends from the FORTH implementation.
    6. Run, test and debug the application on the target.
  • Example:
    • FORTH on an embedded board with 808x / Z80 / 6502 CPU.

Classical - 100% of static Code Generation on the Host, optional dynamic Code Generation on the Target

  • The FORTH system boots on the target.
  • Typical in the era of software updates on the target, by buring new EPROMs ( 1975-2000 ).
  • All modern software tools for editing, code generation and source code saving can be used. But at that time, 1975-2000, typically no CVS. No continous delivery though.
  • Waterfall software development process:
    1. Part I.
      1. Create source code on the host, using a simple editor or an IDE.
      2. Generate code on the host, using an IDE, compiler, assembler.
      3. Optional testing on a hardware & software simulator, on the host.
      4. Burn EPROMs for the target, or flash the complete FlashMemory of the target.
    2. Part II.
      1. The host provides an editor for creating & editing source code.
      2. The target is connected to a serial terminal of a host.
      3. The host transfers FORTH source code to the target, as if typed in by humans. The transfer stopps automatically in case of syntax errors or other faults, deteced by the FORTH system on the target.
      4. The assembler is integrated part of the FORTH system on the target.
      5. New words may be implemented immediately, but just temporarily.
    3. Part III: Run, test and debug the application on the target.
  • Example:
    • FORTH on an embedded board with 808x / Z80 / 6502 CPU.

Primitive Classical, on the Target

  • The FORTH system boots on the target.
  • The target provides an editor for creating & editing source code.
    • Either the target has a keyboard input and video output, or the target is connected to a serial terminal of a host.
  • The assembler is integrated part of the FORTH system on the target.
  • New words may be implemented immediately.
  • FORTH definitions may be edited in the dictionary on the fly, rather than using “Screens” as most Forths of that era did. I.e. there is no saving of “Screens”. By this, source code was not stored, but immediately converted to “Block Code”, which could be re-edited. Remember that there was just 1 KByte of RAM!
  • No CVS.
  • Saving of the the code ( new words, the application ) on an external serial storage medium, i.e. cassette recorder tape.
  • Example:
    • FORTH on the personal computer “Jupiter ACE” / “Jupiter Ace 4000”.
    • FORTH on the personal computer “JU+TE-FORTH-System”.

Modern & Advanced Classical, on the Target

  • An operating system system ( or the FORTH system ) boots on the target.
  • Either the target has a keyboard input and video output, or the target is connected to a serial terminal ( on a host ).
  • The target provides an editor for creating & editing source code.
  • The assembler is integrated part of the FORTH system on the target. So mostly the assembler is no independant executable application. I.e. even if there is an operating system like MSDOS, the assembler ist mostly not an executable file on the file system.
  • New words may be implemented and made permanent immediately.
  • Typically no CVS.
  • Saving of the source code and the code ( new words, the application ) on the target.
    1. By a 16 line x 64 character / line block system. It is not transparent to the user, how this block data is stored.
      1. In files on a file system. Today, most forths load just plain text files.
      2. As raw data on a storage media, e.g. sectors on a floppy disk. Implemented in FORTH systems of the 1970th and early 1980th - This is a feature from the past which made it easy to get started, e.g. on a target with floppy disks, but without affordable operating system, e.g. on CP/M hardware without expensive CP/M license.
    2. On a file system, FORTH sources as text files, FORTH binaries as binary files.
  • Examples:
    • FORTH on the CP/M, Apple II+, MSDOS platforms with disk drives, ESP32forth with on-board SPIFFS file system.
    • FORTH with block system ⇒ See “The FORTH 64×16 Screen Paradigma”.
    • FORTH with file system ⇒ See “The FORTH 64×16 Screen Paradigma”.
    • ESP32forth provides a simple editor by the serial terminal.
    • FORTH on TI-99/4A,... “TurboForth supports disk blocks for storing program code and data (or indeed, anything you wish to store) on disk”.

Modern, on a rich Target

  • Target is a desktop PC, notebook PC, server or an embedded system, running a rich operating system, e.g. Linux, Windows, MacOSX ⇒ An operating system system boots on the target.
  • All modern software tools for editing, code generation, continous delivery and source code saving ( by a CVS system like Git / GitHub ) can be used.
  • New words may be implemented and made resident, immediately.
  • The editor for creating & editing source code must not be integral part of the FORTH system, but you may use an external editor.
  • Examples:
    • ForthWin / Forth2020 - Uses Notepad++ as editor. The documentation describes how to use another external editor.
    • Gforth - Uses Emacs as editor.
    • WinForth - Has it's own editor “IDE”.

Modern, on the Host but with an embedded Target

  • The FORTH system boots on the target.
  • The source code is created on on the host, by an IDE or a rich text editor ( e.g. Visual Studio Code,... ), which has special plugins to support FORTH software development. Saving of the source code on the file system of the host.
  • All modern software tools for editing, continous integration and source code saving ( by a CVS system like Git / GitHub ) can be used, on the host.
  • Compilation process:
    1. On the target.
      • The created source code is transfered to the target, by a special terminal application on the host ( RS232 terminal emulator on the host, with special text file transfer options ).
        • On the target, there is just a “FORTH prompt” waiting for new commands. So that a host IDE must carefully - hopefully with RS232 handshake - transfer the text of a FORTH word definition as if typed by a human.... :-(. This is legacy style of software development :-(.
        • Problem #1, if there is no handshake... at all :-(.
        • Problem #2, if there is handshake, but the limited serial buffer is not large enough, or the host terminal gets a timeout if it must wait too long. So that the FORTH system needs precious time to compile a word... while the dumb host IDE tries already to push FORTH source code to the target... :-(.
      • On the target, the FORTH system compiles it to code and loads the new words into RAM. The assembler is integrated part of the FORTH system on the target.
      • The updated FORTH system may be stored as dump on the host, and / or it is stored on the permanent FlashMemory memory on the target. But at the end of the day, you have to transfer a FlashMemory memory dump back to the host ( by an RS232 terminal emulator on the host, with special binary file transfer options ).
      • By this, new words may be implemented and made permanent immediately.
      • Example: IDE “4E4th - Forth For Education” for MSP430 Launchpad.
    2. On the host.
      • Experts told me: Most C-FORTHs can be said to be crosscompiled. Some may have ways to append “compiled” forth words to that source in some data form. Either just as source or as generated code/data.
      • By this, new words may be implemented and made permanent immediately.
      • The Ardunio IDE works similarily, but there the code is totally generated on the host.

Resources

The FORTH 64x16 Screen Paradigma

FORTH with forced 64x16 Screens only

  • The 64×16 “Screen” ( also called “1 Kbyte Block” ) was the a completely normal Forth model, until the late 1980s. The FORTH “1 KByte Block” was the equivalent to the block size on permanent storage systems, e.g. the Floppy Disk.
    • A “Sector” on a standard Floppy Disk = 512 Bytes.
    • The “Cluster Size” of two standard Floppy disk sectors = 1024 bytes.
    • Classical “Screens” of legacy historical FORTH systems don't have end-of-lines ( i.e. CR/LF, CR, LF ).
  • Source code comments:
    • In times of little RAM, there was no space and so no option for source code comments.
    • In the first line of a 64×16 Screen.
    • Source code comments since about 1980, by “Shadow Screens”.
      • Most keyboards, also of the first personal computer generation since 1977, are equipped with arrow keys, either two ( Apple II+ ) or four ( IBM-PC, ATARI-ST, AMIGA, MacOS,.. ).
      • If the FORTH Screen editor displays a “Screen”, you may switch to the “Shadow Screen” by pressing the arrow key ( ”⇒” ).
      • The “Shadown Screen” is a 64×16 block just for comments, no code.
      • If the FORTH Screen editor displays a “Shadow Screen”, you may leave to the “Shadow Screen” by pressing the arrow key ( ”⇐” ).
      • User#2 “There was even a third type of code documentation, the shadow screen. Forth code was originally written in 64-character by 16-line screens, stored on disk as numbered blocks. Blocks 1000 to 1999, say, were assigned to interpretable Forth code. Then 2000 to 2999 were documentation screens. I.e., 1000 (in this example) above every code screen was its documentation screen. / / Since more complex file systems are used today, shadow screens are no longer necessary. –
      • User#3 “Those must have been pretty ancient, or “tiny” (as in “tiny C”) versions. Even FIG-Forth in 1978 supported both parenthetical and backslash comments”.
      • User#2: “I'm talking 1973/74. Well before '78 we'd blacklisted Forth as an unusable language”.
  • How was the most simple 64×16 screen editor implemented?
    • User feedback: “figForth for the 1802microprocessor needed a low level coded EMIT for output ( It could be an external terminal or some internal program giving the EMIT output on a screen ) and KEY to fetch keyboard input. These Forth words should be available in any Forth system. I expect the Forth editor was pubished somewhere, perhaps in Forth Dimensions.

List of FORTH Systems, categorized by the Editor Modes ( Blocks, Streaming Files )

Legacy 64x16 Screen-only FORTH
  • PC/FORTH by LMI, an MSDOS FORTH. It contains an excellent editor for block screens (64 chars, 16 lines). The top line was reserved for a screen comment.
  • volksFORTH MS-DOS (Intel x86 Architecture i8086-ia64), volksFORTH 6502 (Commodore 64, Commodore Plus 4, Commodre C16, Atari XL/XE, Apple I), volksFORTH Z80 (CP/M, Amstrad CPC), volksFORTH 68000 (Atari ST).
    • volksFORTH 6502: Screens of 24 lines a 41 characters + 1 line of 40 characters to fit the C64 screen.
  • Z79Forth.
FORTH with Screen & processing of standard Textfiles ( "Stream Files" )
  • ESp32forth hast a built-in “1K Screen” editor, but accepts a seamless character stream ( = the contents of a standard text file ) by a terminal connection.
  • Win32forth has a special editor which can handle both “Screens” and standard text files.
  • Gforth still comes with a block editor, PolyForth-style.
    • Brief operating instructions for the block editor:
      use blocked.fb
      1 load
    • and you have a block editor similar to the PolyForth editor ( of course, the block editor is in a block file, the only one in the Gforth distribution ).
      0 l
    • will give you the help screen. The last line there says “If you don't like it, write a block editor mode for Emacs!”
  • bigForth has a GUI-based editor, which is also capable of editing blocks.
  • Notepad ++ can be configured to operate in a 64 columns mode.
No Screens anymore
  • SwiftForth.
Resources
  • In the early 1990s, the editors of most of the FORTH implementations were converted to text editors without this restriction. Newly implemented FORTH systems were designed to accept text files. The “Screens” remained as an option.
  • Expert feedback in 2021: “It would appear virtually no-one uses Forth block editors these days unless it's the only option or they are seeking a serious retro fix”.
FORTH Systems with standard Text Files, without 64x16 Screens
  • Source code comments:
    (  ... )
    \  ...
    

Excursus: Use of SQL Databases to store FORTH Screens

Maximum Length of Strings in SQL Databases
  • IBM DB2.
      • Maximum length of CHAR (in bytes or OCTETS) 255.
      • Maximum length of CHAR (in CODEUNITS32) 63.
      • Maximum length of VARCHAR (in bytes or OCTETS)2 32,672.
      • Maximum length of VARCHAR (in CODEUNITS32)2 8168.
      • Maximum length of LONG VARCHAR (in bytes)1 32,700.
      • Maximum length of CLOB (in bytes or OCTETS) 2,147,483,647.
      • Maximum length of CLOB (in CODEUNITS32) 536,870,911.
  • Microsoft SQL Server.
    • SQLShack "SQL varchar data type deep dive", 2019 - “Microsoft SQL Server 2008 (and above) can store up to 8000 characters as the maximum length of the string using varchar data type. SQL varchar usually holds 1 byte per character and 2 more bytes for the length information”.
      • LOB Type - SQL Server Data Type - Max. Size.
      • BLOB - varbinary(MAX), Image - 2,147,483,647.
      • CLOB - varchar(MAX), Text - 2,147,483,647.
      • CLOB – Unicode nvarchar(MAX), NText - 1,073,741,823.
      • XML data - xml - 2,147,483,647.
    • The Text, NText, and Image data types come from the previous versions of SQL Server.
  • MariaDB.
      • MariaDB "VARCHAR" - “A variable-length string. M represents the maximum column length in characters. The range of M is 0 to 65,532. The effective maximum length of a VARCHAR is subject to the maximum row size and the character set used. For example, utf8 characters can require up to three bytes per character, so a VARCHAR column that uses the utf8 character set can be declared to be a maximum of 21,844 characters”.
      • “VARCHAR(size) Maximum size of 255 characters” - Limit of legacy versions?
  • MySQL.
  • SQLite.
Technical Problems to store a FORTH Screen by an SQL String in SQL Databases

Resources

    • User A: “The rule, a Screen 'does not under any circumstances whatsoever allow a definition to span a block boundary' was not always true. Some old fashion Screen/Block-based FORTH systems of the 1980th have an immediate word
      -->

      that allows 'continuing compilation, i.e. LOAD-ing, to the next Screen/Block”.

      • The definition of
        -->

        implemented in these three versions, that effectively allows a colon-definition to cross a Screen/Block boundary, is as follow:

        : -->
        ?LOADING
        0 >IN !
        B/SCR BLK @ OVER MOD - BLK +!
        ; IMMEDIATE
      • Some FORTH developers use the Term “Screen/Block”. But there is not always a 1-to-1 correspondence between “Screen” and “Block”.
        • A “Block” refers to a memory area (512 bytes usually) that is the image of a disk sector or an equivalent chunk of a plain file.
        • A “Screen” refers to a set of contiguous blocks that can be interpreted via LOAD (usually B/SCR is 2).
      • In addition ?LOADING has #22 error-message usually associated with it
        : ?LOADING
        BLK @ 0= 22 ?ERROR
        ;
      • -->

        is deprecated in the 2020s. It is replaced by the standard word THRU.

      • User B: “My recollection was that
        -->

        could be used BETWEEN word definitions, but it could not be used IN THE MIDDLE of a word definition. The implication, if I understand it correctly, was that a SYSTEM, a SET of word definitions, could span a block, but NO ONE WORD DEFINITION could cross a block boundary”.

How to organize a FORTH Project ?

Prologue

  • I know that in classical FORTH times of the 1980th, with little RAM, where a project must fit into 1 Kbyte of RAM ( Jupiter Ace !), or little more, it must have been easy to keep the overview over a FORTH project. But today with a simple ARM or MSP430 CPU, you have such an amount of interfaces and hardware modules, and with now a little bit more RAM ( e.g. 64Kbytes,..) anf FlashMemory ( e.g. 256 KBytes,..) so that the applications get more complex.
  • You have definitely this structure in a modern embedded software system:
    1. A driver level of C ( in case of a C-FORTH ) or assembly language ( as FORTH words implemented in assembler ).
    2. A lowest function level in FORTH which calls the drivers.
    3. Middleware.
    4. Top of application, e.g. user interface and control of the business process, technical process and / or the device.

Hierarchical tree, BottomUp

  • You first describe some basic FORTH words, and then build higher FORTH words until you reach a control level, where the application is really-working. You create the API before you are using it.
  • This seems to me the most “obvious” style for FORTH.
  • The same style is used by agile web developers, who don´t start with a defining a database model but prefer a non-SQL database as they can later add some fields as soon as needed.

Hierarchical tree, TopDown

  • The usual style for non-embedded applications, also using mockups, rapid prototyping tools &methods: First you bild the top of application, e.g. user interface and control of the business process, technical process and / or the device, but there is still no “action”. For the lower functions, you create an API before you implement it.
  • How can this be done by FORTH, as FORTH functions are based on earlier-defined FORTH functions which give results and have parameters...?

Hierarchical tree, first Drivers then TopDown

  • The most common and simplest way to create embedded systems is to first implement a basic low-level API of functions which initializes the hardware for the 1st. software release and gives access to the needed hardware for 1st. software release, and then switch to topdown development. So that you quickly have access to the hardware ( blinking LED,..), so that the top-level application can operate with limited functions quickly.
  • If the hardware manufacturer of the embedded board provides a “board support package” ( EN.Wikipedia "Board support package" to be loaded into RAM ) or even a “BIOS” ( EN.Wikipedia "BIOS", DE.Wikipedia "BIOS", usually resident in ROM or FlashMemory ), these implements hardware initalisation routines and non-reentrant low-level drivers for the board.
  • This is called in modern terms “Minimum viable product” ( EN.Wikipedia "Minimum viable product", DE.Wikipedia "Minimum Viable Product" ).
  • With FORTH, this might work if the top application code uses kind of mockup to pass by middle-level application functions, but that for a rapid prototyping toplevel functions call lowlevel functions.

How to document the Structure of the FORTH code of a FORTH project, inside the FORTH system or even outside ?

  • Documentation of the Structure of the FORTH code of a FORTH project:
    • HolonForth - “The first Holon system had a structure of modules, groups and (Forth) words”, inspired by the hierarchical presentation of classes in Smalltalk IDEs.
    • You might use graphical tools ( UML? ), or simply sketch tools to document the hierarchical tree of your FORTH vocabulary.

FORTH, a Spaghetti Code Language ?

  • Who agrees that in “old times”, or with hobbyist project even still nowadays, FORTH programmers are known to practice this style?
  • Usually this style is accused to be practiced by BASIC or C programmer, but if you do BottomUp programming in FORTH you loose any control what you are doing, easily...

Resources

IDEs & Editors with FORTH Support

4E4th - Forth For Education

Atom

BBEdit

  • 8th is shipped with syntax file for the MacOSX-only editor BBEdit, in the ”/docs/” folder.

Emacs

MyFORTHshell

Notepad ++

Notepad ++
AceFORTH Syntax Highlighting ( NPP_AceFORTH_Syntax )
Plugin "TextFX Characters"
Python Script by Claudia Frank
Using Regular Expressions to split after 64 characters, by tomas-chrastina
Resources
  • “Settings / Preferences / Language” - Language Menu, Available items: 'Forth'” - Syntax highlighting of words versus numbers :-), no line break after 64 characters :-(.

UltraEdit

Vim

  • 8th is shipped with syntax file for Vim, in the ”/docs/” folder.

VisualStudio Code

The IDE & FORTH
The IDE & Debugging ( C, FORTH, ... ) on ESP8266

Resources

  • For some editors and IDEs, it is possible to limit the line length to 62 characters + CR/LF, to emulate FORTH screens. But even if the line length is limited to 64 characters -including CR/LF -, this does not provide 1K text blocks ( due to missing space characters after the last non-space character, due to the missing limit to 16 lines ).... :-(.

Terminal Technology

Operate FORTH by Terminal

Terminal Software with FORTH Support

e4thcom
escom
  • GitHub "Edzelf / escom" - “Communication for small Forth devices. Like e4thcom, but it will run under Windows”.
forthtalk
MyFFshell

Standard Terminal Software & USB Serial Drivers / Virtual COM Port (VCP) Drivers

Protocols to transfer Data between 2 Terminals

Resources

FAQs, Help, and Tutorials

FORTH Sample Code & Sample Programs

Project Forth Works ( 2023 )

Coding Snippets

Coding Tasks

Simple Projects for FORTH Starters

Resources

Resources

Forums, Newsgroups

Appropriate OpenDirectory Directory Pages

 
en/forth03.html.txt · Last modified: 2024/04/19 19:55 (external edit) · []
Recent changes RSS feed Powered by PHP Valid XHTML 1.0 Valid CSS Driven by DokuWiki