Ever wanted to take a SPARC for a spin? Appraise an AlphaServer or inspect an Itanium? Are you a developer who received a bug report on some obscure platform you’ve never heard of before? Or simply one who cares about portability, and would like your software to support platforms to which you have no direct access?
The ISA Sandbox is an environment which allows developers and users free shell access to bare-metal hosts of unusual, obscure and/or obsolete hardware to which they are unlikely to have access otherwise. There are no special entry requirements in order to be provisioned an account, simply an interest in alternative CPU architectures is sufficient.
The ISA Sandbox is a shared-usage environment, where user resource consumption is not limited by cgroups or any other method. It is very possible to kill a system by forkbombing, consuming all available memory, etc. intentionally or unintentionally. Some of the available systems are naturally very resource-constrained due to their age. Therefore it is asked that you be considerate of other users’ usage on the system. If possible, if a system is already under load, try to limit usage to half of the available cores (where applicable). Limit your memory usage or ask for advice if you see the OOM killer being invoked on your or somebody else’s processes.
Network traffic is on an unlimited symmetric gigabit connection (10 gigabit between hosts). More importantly, traffic is associated with static egress IPs assigned to my business account. Despite being relatively restricted, it is still possible to abuse this availability (DDoS, scraping, proxy for personal traffic, etc). Do not treat the Sandbox as your personal VPS. ABUSE OF THE NETWORK WHICH IMPACTS MY IP REPUTATION WILL RESULT IN REMOVAL OF OUTBOUND NETWORK PRIVILEGES.
User home directories are shared across all machines, using Samba over SMB1 with Unix extensions. The Samba encryption extension is also enabled. Home directories are mounted on-demand, via AutoFS, when logging in. This means that all files under a given home directory are required to be owned by that user. Attempting to chown anything in a home directory will fail. chmod will still work.
Because of this restriction, authorized SSH keys are stored on the root filesystem, in a user-writable file, at /usr/local/share/authorized_keys/%u. If you need to add additional authorized keys, it must be done manually across each host which requires the additional key.
Each user is provisioned 10GB of space initially (unless they request more in their provisioning message). You can also request any reasonable amount of additional space, and it will be granted - justification not required except for exceptionally large requests.
Running a public shell environment is incredibly risky. Strict security measures are enforced to the highest degree possible. The restrictions which may be visible to an end-user are:
sudo) privilege by default. Unlike getting access to the system, being granted root privilege requires justification, except for well-known free software developers: kernel maintainers, mainstream distro developers, GNU or LLVM toolchain maintainers, or employees of free software companies representing their organization in a professional context. Assuming root privileges usually comes with the expectation that exclusive access to the machine in question is required, and so other users will be denied access until the work requiring root privilege is finished. Root privilege also comes with access to out-of-band management resources, where possible.uptime(1) and free(1) can be used to estimate the current load on a system, or the equivalent monitors in htop(1)./etc/resolv.conf on any host. NTP traffic is required to use NTPS. DNS traffic is unencrypted internally, but uses DNSCrypt for communicating with recursive resolvers over the internet./usr/local/share/ca-certificates/ca.crt on any host. The CN for this certificate is ISA Sandbox TLS intercept.umask of 027 is set by default for all users. This will prevent other users from viewing your files by default, but still make it possible to provide read-only access to particular other users by adding them to your primary group.SSHFP records are provided in the public DNS namespace for validation of SSH host keys.
Outbound connections support IPv4 and IPv6, but inbound connections support IPv6 only! You MUST have IPv6 connectivity in order to connect to any host (you can forward through a jump host with IPv6, such as a VPS if your local network does not have IPv6). Public IPv4 records will return 0.0.0.0 for all hosts rather than NXDOMAIN due to a limitation in BIND. You can force your SSH client to use IPv6 with the -6 command-line argument, or by setting the following in your client config file:
Host *.matoro.tk
AddressFamily inet6
| Hostname | Machine | CPU | RAM | Local Disk | Network |
|---|---|---|---|---|---|
| sparcdev.matoro.tk | Oracle SPARC T4-1 | Oracle SPARC T4 (8c/64t) | 32GB DDR3-1333 ECC | 4x Hitachi 300GB 2.5" SAS (hardware RAID5) | Intel X520-DA2 10G SFP+ |
| ia64dev.matoro.tk | HP Integrity rx2800 i2 | 2x Intel Itanium 9340 (4c/8t) | 16GB DDR3-1033 ECC | HP 146GB 2.5" 15K SAS | Intel X520-DA2 10G SFP+ |
| riscvdev.matoro.tk | SiFive HiFive Unmatched | SiFive FU740-C000 (4c) | 16GB DDR4-1866 | Samsung 970 EVO Plus 250GB M.2 NVMe | Mellanox ConnectX-3 10G SFP+ |
| mipsdev.matoro.tk | Cavium Octeon II CN68XX EVB | Cavium Octeon II CN6880 (16c) | 8GB DDR3-1333 | Lexar 64GB UDMA7 Compact Flash | Mellanox ConnectX-2 10G SFP+ |
| alphadev.matoro.tk | HP AlphaServer DS15 | DEC Alpha 21264C (1c) | 2GB RDRAM-133 | HP 73GB 3.5" 10K SCSI | Broadcom BCM5703 1G |
| hppadev.matoro.tk | HP Integrity rp3440 | HP PA-8900 (1c/2t) | 8GB DDR-133 ECC | HP 73GB 3.5" 15K SCSI | Broadcom BCM5703 1G |
| ppc64dev.matoro.tk | Raptor Computing Talos II | IBM CP9M36 POWER9 v2 (paravirtualized) (8c) | 12GB DDR4-2666 ECC | 128GB qcow2 on Samsung PM981a 1TB M.2 NVMe | Intel X520-DA2 10G SFP+ |
| ppc64ledev.matoro.tk | Raptor Computing Talos II | IBM CP9M36 POWER9 v2 (paravirtualized) (8c) | 12GB DDR4-2666 ECC | 128GB qcow2 on Samsung PM981a 1TB M.2 NVMe | Intel X520-DA2 10G SFP+ |
| loongdev.matoro.tk | Loongson 7A2000 | Loongson 3A5000 (4c) | 8GB DDR4-3200 | Samsung PM981a 256GB M.2 NVMe | Intel X520-DA2 10G SFP+ |
dmesg(1). The only way to clear the state is with a reboot. This affects all PA-RISC hardware, and is likely a kernel or glibc bug. If you can figure out what causes the issue - here is the hardware to use for it!Obviously, there are many use cases where root, or at least a simulation of root is required. One example is running a complete system, such as a Debian ports environment. I’ve provided a combination of tools that can get very close to approximating this, with some caveats.
First, create a blank image of the relevant size:
$ dd status=progress if=/dev/zero of=mygentoo.img bs=100M count=30 oflag=dsync
$ mkfs.ext4 mygentoo.img
Then, enter a namespace with isolated users, mounts, and PIDs with yourself mapped to the root user.
$ unshare -mpfr --map-auto
Use fuse2fs(1) to mount the blank image.
$ mkdir mygentoo
$ fuse2fs mygentoo.img mygentoo
Proceed with the installation. You still will not have permission to call mknod(2), so you’ll have to exclude /dev. For Gentoo this might be something like:
$ cd mygentoo
$ tar xvJf ../stage3-*.tar.xz --xattrs-include='*.*' --numeric-owner --exclude='./dev/*'
For Debian, this might be something like:
$ FAKECHROOT=true debootstrap --variant=fakechroot sid mydebian/ http://ftp.ports.debian.org/debian-ports
Now, you will have permission to bind-mount in the pseudo-filesystems while in this namespace. You won’t be able to write to them (unless they are world-writable outside the namespace).
$ mount -t proc /proc proc
$ mount --rbind /dev dev
$ mount --rbind /sys sys
$ chroot . /bin/bash
$ source /etc/profile
Now you are simulating a chroot as closely as possible without root permissions. There are several known issues:
root inside the namespace (which translates to your user outside the namespace). Anything which attempts to be performed as a non-root user inside the namespace will fail. For Gentoo, builds must be performed with FEATURES="-userpriv -userfetch".ext4 driver. Unfortunately I have yet to find a way to allow regular users to use the kernel driver without exposing trivial escalation to root. However, if you need root-like capabilities, this is the only way to do it.I really need root for (xyz reason), pretty please?
See above, “chrooting technique”. If this is insufficient (you need to test kernel patches, etc), reach out on IRC etc. The answer will likely be yes only if you have a well-known history of kernel contributions, or are a known developer for a core free software package.
I am getting weird “permission denied” errors in my home directory.
Home directories are mounted over CIFS. chown is not permitted, not even to yourself (because all guests are translated to nobody on server side). Rewrite your scripts, or see “chrooting technique” for a workaround, and beware the performance overhead.
Samba is just too slow in general, I need real local disk access.
This can be granted on a case-by-case basis. Reach out and request it with justification (along with which hosts you need it on, since disk space is very tight on some hosts).
I need (xyz package) to be installed, can you install it please?
Reach out, if it’s generally useful (or a pain to compile locally rather than system-wide) it can be added to the standard package set.
Can you enable/disable/change some kernel or sysctl config?
Same as above, likely yes.
I discovered a security issue that allows me to access your LAN, escalate to root, gain shell access to one or both of the infrastructure hosts (
matoro-serverormatoro-storage), bypass the packet capture, or do something else that might be considered naughty.
Please reach out ASAP to let me know.
Why is my username
g-mycoolusernameinstead ofmycoolusername?
All guest users are in the guestusers group (GID 29999), but I will also prefix all guest account usernames with g- in order to be able to quickly visually distinguish guest users from system users.
I previously had access to one or more hosts prior to public release, and now my username doesn’t work, what gives?
All users that previously had access still have it, but your username has been changed to include the g- prefix. Adjust the username you use to log in. Also, if you previously had sudo permissions, they have been revoked, so if you still need them, please reach out on IRC.
How do I know how much disk space I have remaining?
Run this command (example output included):
$ df -h ~
Filesystem Size Used Avail Use% Mounted on
//syslog.matoro.tk/guest-testuser 10G 1.3G 8.8G 13% /media/guest-homedirs/testuser
The Size, Used, and Avail fields should be self-explanatory.
My commands are failing with
No space left on device.
You are out of disk space. Use this command to investigate what files are taking up the most space:
$ ncdu -x ~
Then, if you really need it, ask in IRC for more space.
As the ISA Sandbox is a hobby project, and also costs me $200 USD a month in electricity alone, there are NO GUARANTEES OF AVAILABILITY WHATSOEVER. All hosts can and will go down for any duration including indefinitely for any of the following reasons:
Have a question that isn’t answered above? Come join the IRC channel at #isa-sandbox on Libera.chat. For now this is also how to join, provide your desired username and public SSH key.
Alternatively, click on the following link to send a signup request email. Be sure to include your desired username and public SSH key. Signup request