QNAP NAS

I replaced my ancient (but working) DLink NAS with a much newer and faster QNAP NAS (Model TS-464-8G). The QNAP hardware is nice: a compact package supports 4 SATA drives in a variety of RAID configurations, 2x NVMe drives, 2x 2.5GbE ports with option to add a 10GbE card, has a slick web-based user interface, and consumes relatively little power. It runs a custom linux on an a Celeron N5095.
I don’t like the custom linux.

UPS Support

Naturally, I want my data storage to be protected by a UPS and to automatically and safely shut down before the UPS battery is exhausted if there is an extended power outage. I use a CyberPower CP1500PFCLCD UPS (which I am very happy with so far) to protect several NUC servers, an L2 switch, and the NAS. The UPS is connected to one of the NUC proxmox servers via USB. I run NUT on that server, including the nut-server that allows other machines (such as the NAS) to access the UPS over the network as nut-clients. The problem is that QNAP makes this more difficult than it has to be. They support nut (which is nice), but they seem to have done so mainly to allow one QNAP NAS to access another QNAP NAS connected to the UPS.

This is what I had to do to get the QNAP NAS to run as a generic nut client:

  • Control Panel -> External Device -> UPS Tab
  • Select Network UPS Slave
  • Enter the IP address of your nut server
  • Apply changes
  • Reset the NAS to start the upsutil (nut-client daemon) running

How did the NAS get the NUT UPS name, user name, and password used on the nut-server? It didn’t; the NUT support from QNAP hard-coded them as ‘qnapups’, ‘admin’, ‘123456’. And folks wonder why QNAP has had security issues.

You can change the user name and password by enabling the admin user, logging into the NAS via ssh as the admin user, and editing /etc/config/ups/upsmon.conf (make a .orig copy first). Find the line that reads:
MONITOR qnapups@myNutServerIp 1 admin 123456 slave
and replace ‘admin’ and ‘123456’ with the user name and password you have assigned for slave devices on your nut server in /etc/nut/upsd.users

Unfortunately, QNAP doesn’t let you change the UPS name; it *must* be qnapups
Fortunately, NUT provides a workaround for this that doesn’t require changing all the other nut clients. On your nut server, edit your /etc/nut/ups.conf file and add a new dummy UPS named qnapups that points back to your real UPS. For example, my ups.conf ends with:

[cp1500]
    driver = usbhid-ups
    port = auto
    desc = "CyberPower CP1500PFCLCDa"
    vendorid = 0704
    productid = 0601

[qnapups]
    driver = dummy-ups
    port = cp1500@myNutServerIpAddress
    desc = "Proxy UPS for QNAP NAS"

Restart the nut-server (sudo service nut-server restart) and voila your QNAP can then see the UPS:

Ubiquiti UISP

Ubiqiuti is a well known manufacturer of pro-sumer/small-business networking gear. They make two main lines of equipment: UniFi and UISP. The former aims for centralized control only (you can only manage devices through their management software); the latter is more traditional and allows for both direct device management (via command line and web interface) as well as centralized management through their free network management software UNMS. I’m old fashioned so I use the latter.

I use three primary types of gear: an EdgeRouter-X serves as the primary gateway into my network, a variety of EdgeMAX intelligent (layer 2) switches form the wired backbone of the network, and AirCubes provide wireless access.

The EdgeRouter is a particularly remarkable value; at $59, it provides a very full-featured comprehensive router + 4-port GbE switch. With hardware acceleration enabled, it delivers roughly 107MB/s (i.e. it routes at full gigabit speeds) while providing extensive support for features like VLAN, ipsec, dhcp management, NAT routing, etc. It has many more advanced features that I don’t presently use.

The EdgeMax switches work well and although they cost more than some other layer 2 switches, they work well and are fully supported by UNMS. One of the main advantages of this is managing firmware updates which is handled for all of the UI devices from the UNMS management console.

The UNMS network management package can be run locally (that’s how I use it) or, if you have at least 10 Ubiquiti devices, can be run on Ubiquiti’s cloud NMS. Although I have more than 10 Ubiquiti devices, I run UNMS locally (on a proxmox VM) for better security/control.

Baofeng Radios

Baofeng is a Chinese radio manufacturer that produces a line of radio transceivers. Starting with their UV-5 series, they became insanely popular because they are dramatically less expensive than competing products. A good HT from Yaesu, Icom, Kenwood, or Alinco costs over $100 and usually a few hundred dollars, but a Baofeng HT can be purchased for less than $20; a price point that was unheard of previously and simply amazing.

I couldn’t resist so I bought a few several years ago. I played with them briefly, confirmed that I could hit my local repeaters in the 70cm band, and then they sat on the shelf. Recently, I became interested in amateur radio again and decided to take a closer look at the performance of the radio. Does it reach its specified transmit power? What’s the receiver sensitivity? Most importantly, part of the responsibility of being a ham licensee is making sure you transmit in compliance with FCC regulations (FCC CFR 47 part 97) is it compliant? If it causes you to lose your license, it was no bargain.

TLDR: it turns out that most Baofeng radios are not compliant with FCC rules for amateur radio. To be compliant (47CFR Part 97.307e), any signals other than the intended (fundamental) transmit frequency must be at least -40dBc AND below -16dBm. Of the 5 Baofeng radio models I tested, only one was compliant. The rest generated unwanted (spurious) signals on multiples (harmonics) of the fundamental that violate the rules. Nevertheless, the GT-5R appears to be fully FCC compliant and an incredibly value. Details of all the radios I tested are below:

GT-5R (not Pro) – PASS!

After evaluating 4 Baofeng models that were not legal to transmit with on the amateur bands, I was delighted that the 5th time was the charm! The spectrum analyzer tells the tale (see pics below). For $18.39 with free shipping through amazon, the Baofeng GT-5R is indeed a dual-band 4W+ radio with proper harmonic suppression that allows licensed amateur radio operators to legally transmit in the 2m and 70cm bands. It’s absolutely insane that they can hit this price point; kudos to Baofeng!

GT-5R transmitting on 2m through 40dB attenuation…clean as a whistle!
GT-5R transmitting on 70cm – clean signal!

Transmit power (on high) was +35.54dBm (3.58W) on 2m and +36.55dBm (4.5W) on 70cm.

The measurements do not include cable or connector loss, but that should be very low at these frequencies with 12″ of RG-316. I confirmed the 10dB and 30dB attenuators were spot on using a calibrated RF signal generator. My total amplitude measurement error should have been (a lot) less than 1dB.

I also stepped through each harmonic with the analyzer zoomed in (50kHz span, 300Hz RBW, noise floor below -50dBm) and saw nothing of interest; the only measurable harmonic when transmitting on 2m was the 2nd which was below -30dBm. I was so pleased that I bought a second GT-5R which was also clean when checked (and had higher output power).

UV-17 Pro GPS – FAIL

Would 4th time be a charm? Nope. I ordered a pair of Baofeng UV-17 Pro GPS based on several internet reviews that suggested *they* would be clean and legal to transmit with. Although it’s a neat tri-band HT, sadly, the story was the same as with the other models I tested previously (below): not legal to transmit with on 2m or 1.25m, OK on 70cm. They’re going back tomorrow. The spectrum analyzer (with 40dB of attenuation in front), shows the 2nd harmonic on 2m is at +19dBm!:

UV17 Pro GPS transmitting on 2m
UV17 Pro GPS transmitting on 1.25m
UV17 Pro GPS transmitting on 70cm

UV-82 – FAIL

I connected my Baofeng UV-82, a dual-band (2m, 70cm) HT rated for 1W low and 5W high output power to a spectrum analyzer through a 30dB attenuator and transmitted on 2m and 70cm; the results and discussion are below:

UV82 transmitting at high power on 145MHz (2m band)

The transmitted power on 2m at the intended frequency was 3.9W which is pretty good, but the harmonics are awful. FCC regulations (47CFR Part 97.307e) state that spurious transmissions must be at least -40dBc (40dB below the carrier level) AND less than 25uW (-16dBm). The UV82 doesn’t even come close. The second harmonic is only -19.45dBc and even the third harmonic isn’t down 40dB. Both are well above -16dBm.

I switched the radio to 70cm and again transmitted at full power:

UV82 transmitting at high power on 440MHz

These results at 70cm are much better: the fundamental is at 4W and the second harmonic is down more than 40dB and is below -16dBm. The third harmonic isn’t visible. So, while there isn’t much margin, the UV82 appears to be good for amateur use at 70cm, but is not compliant in the 2m band.

It seems likely that the UV82 contains a low-pass filter that attenuates signals above 450MHz; which makes the radio compliant in the 70cm band, but is worthless for use in the 2m band due to the 2nd and 3rd harmonics. To use the radio in the 2m band legally, you probably need to install a 200MHz low-pass filter such as Mini-Circuits VLFX-225+ between the radio and antenna.

UV-B5 – FAIL

I also have a pair of Baofeng UV-B5 transceivers. They are also problematic with respect to spurious emissions, but interestingly in different ways from the UV82.

UV-B5 2m transmit on high power
UV-B5 70cm high power

In both bands, only the second harmonic is present with good suppression of higher harmonics. Unfortunately, the second harmonic violates the FCC regulations in all cases, so the UV-B5 can’t be used to transmit legally in either band. Compliance requires spurious to be BOTH -40dBc or better AND -16dBm or lower. I tested at high and low power and the second harmonic was much too high. Note: low power was measured at 1.5W at 2m and 1.25W at 70cm.

  • 2m High Power: second harmonic is -40dBc (good!), but is -4.82dBm (fails compliance)
  • 2m Low Power: second harmonic is at -5.88dBm (fails compliance)
  • 70cm High Power: second harmonic is -34.25dBc and at -0.12dBm. (fails compliance)
  • 70cm Low Power: second harmonic is at -8.08dBm (fails compliance)

This behavior is different from the UV-82; the third harmonic and above are well suppressed in both bands so these radios likely have separate VHF and UHF PAs (or at least harmonic filters). Unfortunately, they do not seem to be adequate and, as far as I can tell, it is likely illegal to transmit with the UV-B5 in either band, even at low power, unless you add additional filtering between the transceiver and the antenna.

The UV-B5s have another serious design flaw: if you leave the battery connected, it will gradually be drained, even with the radio off; so you must store the UV-B5 with the battery disconnected. Note: this is not the case for the UV-82.

GT-5R PRO – FAIL

I contacted Baofeng and they advised that their UV-5R and GT-5R comply with FCC part 97 & 15B. On their website, the GT-5R PRO is also advertised as FCC compliant and is tri-band, so I ordered a pair of GT-5R PRO and was dismayed to see that they too appear to be non-compliant on 2m. The first unit transmits at 3.37W at high power and has a huge spur at the second harmonic.

GT-5R PRO transmit at full power in 2m band

The second unit also transmitted at 3.3W at high power in the 2m band (35dBm) with a somewhat lower spur at the second harmonic of 0 dBm: better, but still not -40dBc AND below -16dBm. Interestingly, when transmitting at low power, the fundamental drops to 1.5W, but the 2nd harmonic actually goes UP by about 4dB!

The GT-5R PRO appears to be compliant in the 70cm band with the fundamental at 4.35W and no spurs above -20dBm.

GT-5R PRO transmit at full power in 70cm band

It’s possible I’m doing something wrong in my measurements; I would have liked to filter out the fundamental with a notch or high-pass filter before measuring the harmonics, but I don’t have the right filter and with the signal already knocked down 30-40dB, I don’t think I’m over-driving the SA. So I returned the GT-5R PROs.

USPS Priority International vs. Quality Control

I ship quite a few domestic packages and am a big fan of USPS Priority Mail service. I occasionally need to send packages overseas and at first glance, USPS Priority International service looks really good: low rates, reasonable delivery times, convenient shipping process. Unfortunately, the reality is different: their international shipping seems to be a disaster.

They estimate delivery to China at 6-10 days. The first package I sent took about 3 weeks. I shipped a second package 4 weeks ago; it reached China and cleared customs in 10 days, but then the tracking stopped. Maybe it will still get to its destination, maybe not, but I couldn’t wait any longer and had to ship a replacement (using a different carrier). USPS seems to lose all control over and visibility once they hand off to China Post.

What seems clear is that USPS International has no quality control. Even though their own tracking system shows the package long overdue, they do not automatically start a search…something any company concerned about quality would do automatically. To try to find the package, I started at the post office where I mailed it. I brought them the printed tracking history and showed them that they advertise 6-10 days and it had been 4 weeks. I was very polite and told them that I love USPS domestic priority shipping but can’t use USPS for international shipping if this is representative of the service. The agent said “I’m sorry you feel that way”….not “I’m sorry we lost your package” and told me to “call this number” (i.e. the human agent wasn’t going to help and as a representative of the organization, accepted zero responsibility for a service failure). This was two quality-control problems: 1) a failure of employees to act as representatives of their company and 2) a disinterest in finding and understanding quality problems in order to improve processes.

A company that doesn’t care about quality creates obstacles to reporting problems. Unsurprisingly, when I called the number, I encountered an automated system designed to maximize customer frustration: it demands you enter a tracking number but won’t accept international tracking numbers and refuses to connect you with a human unless you provide a tracking number it likes. Eventually, by trying different menu paths, I reached a human who took down gobs of information and told me I could call back in 35 days if they didn’t find the package. So after 2 months, I might have the opportunity to spend more time filling out extensive paperwork and finding detailed shipping evidence to recover the shipping cost – of a package that their own tracking system shows that they lost.

That’s not going to happen, of course, I’m simply not going to use USPS for international shipping again and will share the experience with friends to save them from going through the same trouble. That’s what bad quality control does: it causes companies to lose business. I sent a replacement for the lost package via FedEx; it cost significantly more, but like UPS, DHL, and other big-boy shipping companies, they will track the package from start to finish and take responsibility for actually getting it there. USPS domestic shipping is wonderful, but if they want to compete in the international shipping world, they need to get their act together and start paying attention to quality.

915MHz Antennae

I do a lot of work in the sub-GHz ISM band and there are a lot of antenna vendors whose quality varies broadly. Consider the case of two antennae: The first was purchased from AliExpress vendor CS Family. These were sold as 915MHz antennae with 3dBi gain (this).

It’s centered at 910MHz and its best-case SWR is 1.34:1. At the band edges, it offers 1.57:1 at 902MHz and 1.77:1 at 928MHz. This is acceptable; I consider anything better than 2:1 acceptable for a cheap antenna and this meets its spec across the band. The impedance is 75 -j47. It’s not pretty, but it will do the job.

However, compare it to one of my favorite compact antennae from TE/Linx: the ANT-916-CW-HWR-RPS (datasheet) claims 1.2dBi and better than 2:1 VSWR.

This antenna is nicely centered in the band and offers an amazing 1:08 SWR at 915MHz, 1.57:1 at 902MHz and 1.40:1 at 928MHz. Impedance is 46.59 +j0.70. Now that’s pretty!

In fairness, both antennae meet their spec (better than 2:1) and the AliExpress antenna cost only $2.16 whereas the Linx antenna cost $13.50…so I cut the AliExpress guys some slack 🙂

Measurements taken using a NanoVNA SAA-2N with VNA Qt software. I might break out a nicer VNA some time and repeat these tests, but for quick measurements, I’ve found the NanoVNA2 to be remarkably accurate; it’s insane what $110 can buy these days. Note: if you have a NanoVNA2 SAA-2N, the firmware, manual, schematic, and a version of VNA Qt that works with it can be found here. The 1.3.07 firmware (Aug 30, 2022) offers many important features and if you’re running older firmware, you probably want to update (instructions are in the manual). The only thing I couldn’t find a way to do using the LCD UI was port extension, but it’s easy with VNA Qt.

Passwords and Security

Cybersecurity (or lack thereof) is a disaster. Every website in the world is under constant attack by networks of automated hacking robots (bot nets) checking for weak security.

Everyone should use a password manager such as PasswordSafe or BitWarden . These tools will generate a different random password for each account/website you use and store it securely encrypted. You “open” your safe by entering a master password which then decrypts all the stored information. The tools will also let you securely store things like the URL of the website, and some free form information like account numbers. All you need to memorize is one master password (make it something good – an unusual phrase).

Because so many sites use bad security practices, there are now massive databases of hacked usernames and passwords available. This would have been impossible if companies followed best security practices of even 30 years ago. This means you are particularly vulnerable if you use only one or a few passwords for many sites. Even if your password is good, you re depending on the developers of the website you entered that password into for its security.

Using different randomly generated passwords for each website means that even if one site uses bad security practices and is hacked, ONLY your account at that site is compromised, not all of your accounts.

Developers: NIST has issued guidance on password practices that are quite good and everyone who writes software that requires password authentication should read this:

https://auth0.com/blog/dont-pass-on-the-new-nist-password-guidelines

Some of the simple things you can (and must) do that have been common practice since at least the 1990s:

  • Don’t store passwords. Anywhere. Not plaintext or encrypted. You should only store a one-way hash of the salted password. Even ancient hashes like MD5 are much better than storing plaintext or encrypted passwords, but there’s really no excuse for using anything weaker than SHA256 or better these days.
  • Use password spinning: after every N failed attempts, add a small (1-3 second) delay. This effectively prevents brute-force/rainbow hacking.
  • Test new passwords against a database of known hacked passwords. (no “OpenSesame”)
  • Require reasonable minimum password lengths. (no “123”)

Not from the 90s, but probably a good idea these days:

  • If you have a fast and easy way to do 2FA (e.g. biometric), use it.

Some things you should NOT do (and that drive me crazy when I see it):

  • Require users change their passwords frequently – this is just nuts; it drives users crazy and incentivizes them to use simpler and easier to remember passwords.
  • Have special characters users must/can’t use in their passwords. This discourages the use of good random password generation and is even worse without it.
  • Disallow users from seeing the password as they type it (that should be an option)
  • Time-consuming 2FA methods (like text messages or validator apps) for things that don’t need that level of security. It introduces a super-annoying delay in accessing your data/app. Biometric 2FA is OK.

Hacking Oscilloscope Bandwidth

The price of oscilloscopes goes up quickly above 100MHz bandwidth, often placing them out of the budget of hobbyists and small businesses. The analog bandwidth of a scope is defined as the frequency at which amplitude is reduced by 3dB (roughly 30%)

Most oscilloscope lines offer a variety of bandwidth options (at increasing prices) such as 50, 100, 200MHz or 100, 350, 500MHz. In many cases, those are really the exact same scope hardware (the highest bandwidth) but are limited in software to a lower bandwidth. This allows manufacturers to address a broader range of potential customers (they can sell the 100MHz scope to individuals or small companies without undercutting the big margins of their 500MHz scope sales to large companies).

Some manufacturers have made it possible for hobbyists to unlock the higher bandwidths. Doing this voids any warranty or calibration of course, and so most labs or large companies simply won’t do this and if you choose to do it, it’s at your own risk. However, I have done this successfully with scopes from Siglent (SDS1104X) and Tektronix (TDS3xxx) and the results have been pretty good.

TDS3K scopes are older Tektronix models that used to cost a fortune (Tek was long the king of the oscilloscope hill and my favorite). The TDS3K line came in 100, 350, and 500MHz models. Fortunately, you can hack any TDS3K scope to 500MHz. For info on how to do this, look to the always excellent eevblog (here). Note that you will need to downgrade the firmware to 3.39 in order to perform the hack. In a nutshell:

  • Install a communications module in the back slot
  • Configure for 9600,8,N,1 (local echo on is helpful)
  • Check current version: *IDN?
  • PASSWORD PITBULL
  • MCONFIG TDS3052 (or whatever model you’re upgrading to)
  • Reboot the scope

Note too that there are downsides to the TDS3K line like calibration is stored in battery backed RAM (and the battery will eventually die). In general, to continue to use a scope of this age, you should plan to re-cap it (replace all electrolytics) and replace the battery-backed RAM module with a new one.

Siglent and Owon are Chinese companies that make a lot of test gear that isn’t quite up to Tek or Keysight standards, but still offer a great deal of bang for the buck and the Siglent products are often hackable. For info on how to hack the Siglent SDS110x scopes you can also refer to eevblog.

An important question is: how do these scopes perform after the hack, so I tested a Tektronix TDS3012 (100MHz) before and after hacking it to a TDS3052. I also tested a TDS3032 I had previously hacked to TDS3052, a Siglent SDS1104X hacked from 100 to 200MHz, and compared them with an unhacked TDS5054B (500MHz) and an Owon SDS8202 (200MHz). In each case I supplied a 0dBm tone from an IFR 2025 RF signal generator through an admittedly less-than-ideal but short coax cable with BNC connectors. For the scopes without internal 50-ohm termination, I used a BNC through terminator and a 6dB attenuator.

A 0dBm signal terminated in a 50R load should be 632mV peak-to-peak. At the bandwidth limit of the scope, that signal should be reduced by 3dB or roughly 30% (-3dBm = 448mV p-p):

MHz:Term1050100200300350400500
TDS3012
(before hack)
Int562320
TDS3012
(hacked)
Int666648644640606580554510
TDS3032 #1
(hacked)
Int680665660644630604578530
TDS3032 #2
(before hack)
Int641627626598563546511457
TDS3032 #2
(hacked)
Int658640637631619604576555
TDS5054BInt662642630627593575560566
SDS1104X
(hacked)
BNC65063860051436026416957
SDS8202BNC680672664608512424300
Peak-to-Peak voltage measured with 0 dBm sine wave

It’s clear that the hacks really do increase the available bandwidth. In the case of the TDS3K scopes, to greater than 500MHz, making them truly the equivalent of TDS305x. I measured a 700MHz signal on a hacked TDS3032 at 466mVp-p so the 3dB bandwidth was even higher than that! The hacked scopes also show the sample rate at 5Gs/s whereas before the hack they top out at 1.25 or 2.5GS/s.

The Siglent SDS1104X (nominally 100MHz) hack also extends its bandwidth to 200MHz; mine was down -1.78dBm at 200MHz and still slightly better than 3dB at 235MHz (-2.88dBm). The trigger was able to lock cleanly out well past 400MHz and measurements remained accurate. Note: the SDS1104X does not offer internal 50R termination so measurements were made through a 6dB pad (reduces signal by half, probe set to 2x) and 50R through terminator.

The old Owon SDS8202 (nominally 200MHz) did remarkably well out past 300MHz, easily outperforming the hacked SDS1104X. Owon makes nice analog front ends! Note that frequency measurement stopped at 200MHz even though the scope was clearly able to trigger on and lock to signals out to 400MHz.

SIM7000x and SSL

I designed the Simcom SIM7000x into a product some time ago; although it is a little dated, it still offers an extensive set of features for a remarkably low price. In particular, it combines a (2G, 3G, 4G/LTE) cellular modem with a GPS receiver and supports very low power modes of operation. The problem is the documentation and I’m publishing this post in the hope that it will save others some pain.

First: the grousing:

Like most communication modules, the SIM7000x is controlled via an asynch serial interface using the venerable AT command set dating back to POTS modems. That so many manufacturers still use this 1981 control interface is remarkable (and so awful that I will post about just this issue in the future). There are three particularly bad things about it:

  1. It tries to combine human and M2M communication and, as a result, is terrible at both. For M2M, there is no framing, error checking, data typing, MIB, easy way to separate tokens, or standard error handling.
  2. The data and control streams share the same serial interface so you can’t (easily) interact with the module while it is transferring data (yeah, I know about CMUX).
  3. Because the interface is so old, has been used by many manufacturers, and several standards bodies have attempted to tame it (each differently), there are now gobs of similar commands that leave a confusing and inconsistent interface which is difficult to code to.

Some day, a smart manufacturer is going to fix this. The SIM7000x has a rich set of peripherals, gobs of I/O, memory, and processor power; it could easily deprecate the AT interface and add something sane. A modern interface would support I2C and/or SPI, layered communication protocol, and standard MIB interface. Nevertheless, the SIM7K modules offer a great deal of functionality, are FCC and PTCRB modular certified (so they’re ready to use on carrier networks), and can be purchased for a reasonable price.

My Applications

Nearly all of my design work involves embedded systems (there are often extensive backend server systems too). For the embedded systems, I typically want to establish a connection to a remote server and exchange data; usually using a RESTful or MQTT API in a format like JSON or msgpack. Because the devices are typically remote from the server, I need:

  • secure communication (TLS)
  • server authentication (public key certificates)
  • remote device authentication that can be revoked

The SIM7000x supports all of this, but the specifics are not always obvious or convenient despite the 281 page AT Command Manual and numerous application notes including TCP/IP, SSL, and HTTP(S) Simcom provides. The manuals should probably be much longer and take more time to explain the API architecture and how the commands. I’ll provide an overview below of how to do secure data communications and include discussion of some of the potential areas of misunderstanding. Note: the discussion below assumes you have the latest firmware installed on the module.

LTE Connectivity

In America, GPRS (2G) and EDGE (3G) data networks have been completely discontinued as of the end of 2022. Most low-cost data connectivity now relies on 4G/LTE which the SIM7000x supports. All the major cellular carriers (AT&T, Verizon, T-Mobile, etc.) support LTE and you can buy SIM cards for any carrier to use with the SIM7K modem module. I use the MVNO Velocity/Flolive; which offers multi-carrier SIMs so the device will connect to whatever carrier is available, anywhere in the world it is deployed (i.e. you’re not tied to one carrier’s network).

Let’s look at the AT commands involved in bringing up an LTE data connection:

Initial Startup of the Modem

  • If you are using the DTR wake/sleep functionality, set the DTR pin to wake the modem
  • Send the modem an AT command and wait for OK response to see if it is already awake
  • If the modem does not respond, try a hard reset wait for the RDY prompt
  • If the modem still does not respond, toggle PWRKEY and wait for the RDY prompt
  • Repeat the above a few times until you get a RDY or OK prompt
  • Reset the modem configuration so you’re starting from a known state using the command:
    ATZ (and wait for OK response)
  • Enable sleep mode if you want low power operation (which I usually do) with the command:
    AT+CSCLK=1 (and wait for OK response)
    (note: this is a persistent setting, you only need to do it once – noted as “AUTO_SAVE”)
  • Configure the modem for LTE operation only. There are many 2G, 3G, and 4G bands to be scanned; by limiting the scan to LTE bands, you can dramatically speed up the time it takes to initially find a carrier (the modem will remember after that). Send the command:
    AT+CNMP=38 (and wait for OK response) (AUTO_SAVE)
    (note: this is a persistent setting, you really only need to do it once)
  • Note: if you know which carrier(s) you will be using, you can further speed up the initial carrier discovery by restricting which LTE bands are scanned to only those supported by the carrier.
  • Configure for LTE-M1 only for the same reason (unless you need NBIoT) using the command AT+CMNB=1 (and wait for OK response) (AUTO_SAVE)
  • Disable the Network activity LED if you wish to save power using the command:
    AT+CNETLIGHT=0 (and wait for OK response) (AUTO_SAVE)
  • Gather and cache any data you want about the modem and SIM card using commands like:
    AT+GMM, AT+GMR, AT+GSN, AT+CIMI, AT+CCID

Register with the Carrier Network

  • Check if the modem has already registered on the carrier network using the command AT+COPS? (look for a response like +COPS: 0 which indicates that the modem is searching for a supported carrier or a response like +COPS: 0,0,”AT&T FloLive”,7 which indicates that the modem is connected via LTE.
  • If the modem is not connecting, you can forcibly cycle the modem through de-registering and re-registering with the carrier using the sequence: AT+COPS=2 and wait for OK then AT+COPS=0 and wait for OK.
  • When the modem connects to the carrier, it can send (depending on some other configuration settings) an asynchronous notice like: *PSUTTZ: 24/06/13,22:40:35″,”-16″,1
  • Once AT+COPS? indicates that you’re connected to the carrier, you can check things like the signal strength with the command
    AT+CSQ (and wait for the response like +CSQ:25,99 followed by OK)

Connect to the Packet Data Network

  • Your data provider may be the carrier like AT&T or a third party MVNO like FloLive that uses many carriers. You will need to know the APN for your data provider to get IP connectivity.
  • You can either hardcode an APN (like flolive.net) or, when using LTE, request the APN from the carrier using the command:
    AT+CGNAPN (and wait for a response like +CGNAPN: 1,”flolive.net”).
    Note that not all carriers will provide the (correct) APN so you may be better off hard coding it.
  • Set the APN using the command:
    AT+CSTT=”flolive.net” (and wait for OK) (not persistent)
  • Check to see if the modem has connected to the packet data network using the command: AT+CEREG? (the response that indicates you are connected +CEREG: 0,1 or +CEREG: 0,5 or are not yet connected +CEREG: 0,2 (searching/trying to attach).
  • Handle critical errors such as +CEREG: 0,0 (modem gave up searching for a carrier) or
    +CEREG: 0,3 (registration denied). Generally your options in these cases are limited to periodically de-registering from the carrier network and re-registering
    (as described above with AT+COPS=2 then AT+COPS=0).
  • Once you are connected to the APN, you can connect to the IP network.

Connect to the IP network

  • Check your IP connection status and IP address using the command
    AT+CNACT? (your response will indicate +CNACT: 0,”0.0.0.0″ if not connected or something like +CNACT: 1,”100.64.132.137″ if you are connected)
  • If you don’t have an IP connection yet, use the command:
    AT+CNACT=1 or AT+CNACT=1,”flolive.net” (and wait for the OK response).
    When the connection is made, the modem sends an asynchronous status message:
    +APP PDP: ACTIVE
  • Wait until you are connected and have been assigned an IP address.
  • You can optionally test the connection by pinging your server or a remote host using: AT+SNPING4=”yahoo.com”,5,16,1000 (ping 5 times with a 16-byte packet and 1s timeout). If you’re connected, you’ll see 5 responses like:
    +SNPING4: 1, 74.6.231.20,194 (indicating a 194ms ping time)

SSL/TLS connection

  • The SSL stack supports several (6) different configurations (contexts). Each context is numbered (ctxindex 0..5). Use the following command to set context 0 to use TLS1.2:
    AT+CSSLCFG=”sslversion”,0,3
  • If your application doesn’t set the date/time on the modem, you can tell the modem to ignore date/time when evaluating the validity of security certificates using the command:
    AT+CSSLCFG=”ignorertctime”,0,1
  • If your server serves multiple domains from the same machine (e.g. apache virtual servers), you can indicate the server name to tell the stack which certificate should be used:
    AT+SSLCFG=”sni”,0,”myserver.mydomain.com”

Configure Connection parameters

  • Configure keep-alive messages depending on your server configuration needs to prevent the connection from automatically closing:
    AT+CACFG=”KEEPALIVE”,1,30,30,1

Establish a secure TCP connection to the server

  • You can configure several simultaneous connections; each is identified via connection ID (cid). Note: the connection cid is different from the context (each cid should reference a context).
  • Close any prior connection and clear connection settings
    AT+CACLOSE=0 (and wait for OK or ERROR response)
  • Configure connection 0 to use SSL:
    AT+CASSLCFG=0,”ssl”,1 (and wait for OK response)
  • If you have installed a server or CA certificate in the modem (see details below), you can configure the SSL connection to authenticate the server using that certificate so it will only connect to the real server:
    AT+CASSLCFG=0,”cacert”,”myCA.crt” (and wait for OK response)
  • If your server certificate is self-signed (typically with a very long expiration date), you may need to tell the stack to ignore the certificate expiration:
    AT+CASSLCFG=0,”ignorertctime”,0,1 (and wait for OK response)
  • Configure the connection timeout:
    AT+CASSLCFG=0,”timeout”,30000 (and wait for OK response)
  • For debugging, you can check the configuration:
    AT+CASSLCFG?
  • Open a TCP connection to the server
    AT+CAOPEN=0,”TCP”,”myserver.com”,443 (and wait for response: +CAOPEN: 0,0)
    (<cid>,<result> where rslt 0=success, >0=fail; note: 24..26 indicate certificate mismatch)

Send data to the server

  • Send your request data to the server (note specification of the number of bytes you will send)
    AT+CASEND=0,186 (wait for a ‘>’ prompt indicating the modem is ready to receive the data)
  • Send the data and wait for the asynchronous completion indication: +CADATAIND:0
  • Request response information from the server. E.g. to request 1000 bytes of response data:
    AT+CARECV=0,1000
  • You can check the connection status if desired:
    AT+CASTATE? (returns +CASTATE: 0,0 (disconnected) or +CASTATE: 0,1 (connected)
  • Close the connection
    AT+CACLOSE=0 (and wait for OK response)
  • Note: to connect again you must again send the AT+CASSLCFG configuration commands; you don’t need to re-configure the AT+CSSLCFG commands; and re-connect via AT+CAOPEN.

Installing a server certificate

For security, it’s important that the communications be sent over a secure channel so they can’t be stolen or corrupted in transit. The TLS connection does this for you using public key cryptography and a Diffie-Helman key exchange whereby the server and your modem agree on an encryption key to be used during the session.

However, it’s also critical that you authenticate the server (i.e. confirm that the server you are communicating privately with is the actual server you intend to be communicating with). This is also done using public key cryptography and certificates. To do this, you must store a public key for your server or for a master authority on the modem and tell the modem to use that public key to check that a signed certificate supplied by the server when you connect to it is valid for the public key you’ve stored in the modem. Only a server that holds the matching private key can provide a proper certificate signature.
A cool feature of certificates is that they can be chained: an higher-level authority can sign a certificate for your server and if you trust that authority, you can trust the server certificate. This means that you can store the public key for that higher Certificate Authority (CA) and then trust any server certificate that has been signed by the CA. The CA is often called the “root of trust”.

Websites typically pay a certificate authority like DigiCert or Google Trust Services to sign their server certificates. Web browsers come with the public keys for most popular certificate authorities so when you connect to a website, the site can automatically be authenticated. Unfortunately, those CA certificates usually have a short lifetime to protect against the possibility that they may be compromised. This is not a big deal since browsers are regularly updated and those updates include new certificates. However, it’s a problem for embedded devices that may not receive frequent updates.

For your own servers and devices, it may be preferable to generate your own certificates (aka self-signed certificates). You can even generate a public/private key pair for your own certificate authority which you can then use to sign and update many server certificates. Then if the public key for your own CA is stored on the modem, it can authenticate any servers you issue certificates to. Your CA can also have a very long lifetime (e.g. 50 years) so that the public key stored in your modem will be valid for the life of the product. The easiest way to create CA and server certificates is using openssl (I’ll make another post about that if there’s interest). To install a server or CA certificate in the modem (both are referred to as a “cacert” in the modem), it must be loaded into the modem’s file system and then converted to a format used by the modem:

  • Initialize a file system buffer:
    AT+CFSINIT
  • Prepare to write the file to the “customer” portion of the modem’s file system (3), 0 indicates overwrite if file existed, 765 is the certificate file size in bytes (change this to the size of your certificate), 2000 is the timeout in ms for the entire download.
    AT+CFSWFILE=3,”myCA.crt”,0,765,2000 (wait for DOWNLOAD response)
  • Send the certificate file in PEM format (starts with —–BEGIN CERTIFICATE—–\n and end with —–END CERTIFICATE—–\n). Note that in most cases, you can get the certificate for your server by visiting the server from a web browser like Firefox. Click on the lock icon, then Connection Secure, then More Information, then View Certificate. There will be links to download the server certificate or the chain of certificates (starts with rootCA) in PEM format.
  • Free the file system buffer:
    AT+CFSTERM (and wait for OK response)
  • Convert the CA certificate format:
    AT+CSSLCFG=”convert”,2,”myCA.crt” (and wait for OK response)
  • You can check the file size or read it to confirm it has been successfully received using:
    AT+CFSGFIS=3,”myCA.crt” (returns +CFSGFIS: 765\r\n\r\nOK\r\n\r\n)
    OR
    AT+CFSINIT (wait for OK response)
    AT+CFSRFILE=3,”myCA.crt”,0,765,0
    (returns +CFSRFILE: 765\r\n … file contents … \r\n\r\nOK\r\n\r\n)
    AT+CFSTERM
  • Note that if you are using the HTTP(S) APIs for the modem (not covered in this post), you can configure them persistently to use this certificate using:
    AT+SHSSL=0,”myCA.crt” (and wait for OK response) (AUTO_SAVE)

Israel/Palestine Primer

Most material discussing the Arab-Israeli conflict constrains the timeframe or geography to advance a particular narrative. This post attempts to provide the longer and broader context needed for a more nuanced understanding.

Jews and Judaism

The Jews are a people (an ethnicity). Ethnicity is a hereditary trait: children born to Jewish parents are Jews; a Jew cannot convert, for example, to being Chinese. The Jewish people originated at least 3500 years ago in the Levant, which is a part of the Middle East, and Jews are indigenous to that region. There are roughly 15.7 million Jews worldwide as of 2023.

Judaism is a religion. Religions are belief systems; you can convert to Judaism or Christianity or Buddhism by changing your beliefs. Judaism originated among the Jewish people and most practitioners of Judaism today are Jews. Unlike Christianity and Islam, Judaism is not a proselytizing religion – there is no religious obligation or incentive to convert others. As a result of this tight coupling of ethnicity and religion, Judaism is often referred to as an ethnoreligion. Today, “Jewish” is commonly used to refer to both the ethnicity and the religion.

Christianity and Islam descend directly from Judaism; both are proselytizing religions. As a result, they have spread rapidly, often through conquest and forced conversion. The unwillingness of many Jews to convert has resulted in extensive persecution, with Jews frequently killed or forced to flee. The dispersal of Jewish refugees globally is referred to as the Jewish diaspora. Today, the large majority of Jews live in either Israel or America.

Ancient Israel

The ancient Jewish kingdoms of Israel and Judah (from which the name Jew derives) covered an area slightly larger than modern Israel and the West Bank (Judea and Samaria). There is extensive archeological record of the people and kingdoms of Israel and Judah starting in the Iron Age such as the Egyptian Merneptah Stele circa 1200 BCE. Jews of this time spoke Hebrew and the kingdoms of Israel and Judah existed for many centuries.

Israel and Judah were conquered by a series of invaders, starting in 720 BCE, including the Assyrians, Babylonians, Persians, Romans, Greeks, Arabs, Turks, and British. While Israel existed as a vassal state under several conquering empires, repeated Jewish rebellions resulted in the Roman empire ethnically cleansing many of the Jews who fled to surrounding countries and eventually as far as Europe. This was the start of the Jewish diaspora. The Romans also changed the name of the region to Syria Palestinia in an attempt to eradicate the Jewish identity.

The Arabs, Islam

The Arabs are a people who originated in the Arabian peninsula in roughly the 9th century BCE. Arab is an ethnicity: children born to Arab parents are Arabs, regardless of their religion. There are, for example, many Coptic Egyptians who are both Arab and Christians. Islam is a religion that originated among Arabs; practitioners of Islam are referred to as Muslims. The Arab leader and prophet Muhammad (PBUH), led an Arab army that conquered much of the Middle East and North Africa to spread Islam and impose Islamic rule. Thereafter, a series of Islamic Caliphates ruled the region for centuries.

Through the Islamic crusades, Arabs spread through much of the Middle East and North Africa. In 2024, there are 22 member nations of the Arab League with roughly 456 million citizens. Nearly 25% of the world’s population is now Muslim. Muslims are the majority religion in 46 countries and in at least 23, Islam is the official state religion.

Palestine and the Palestinians

Palestine refers to a region of the Levant with ill-defined borders (like “the Sahara or the Mid-West”). It has not historically referred to a nation (there has never been a nation named Palestine) but today the Arabs of Palestine aspire to a new nation state named Palestine. The origin of the term is debated, but is likely a Greco-Roman reference related to the ancient Phoenicians who lived in the region. Modern usage is clearer and stems from the Roman era: rebellious Jews were crushed by the Romans who then renamed Israel, Judea, and Samaria as the Roman administrative province of Syria Palestina, which later became just “Palestine”.

Palestine suffered many invasions and conquests. Peoples from all over the Middle East and Europe moved to and through Palestine. Following the 7th century conquest by Arabs and the subsequent Caliphates, many residents converted to Islam and the region generally underwent Arabization. During the periods from 1500-1900, regional famines and conflicts also sparked mass migrations that brought many ethnic Arab immigrants to Palestine.

Exactly who the Palestinian people are today is a hotly contested issue due to its bearing on the conflict. If the population that identifies as Arab is primarily descended from foreign invaders, it weakens that population’s claim to being indigenous. However, if the population is descended from the ancient populations of the region (Canaanites, Phonenicians, Israelites, and Samaritans), it strengthens the case for being indigenous but also strengthens the case for the Jews being indigenous.

Until at least the end of the 19th century century, the term “Palestinian” referred to anyone who lived in Palestine (Jews, Arabs, Christians, Muslims, etc.). The term Palestinian was first used to self-describe Palestinian Arabs in 1898 during the rise of Arab nationalism. However as the population of Palestine was largely Arab at that time, the exclusive usage to denote only Palestinian Arabs remained unclear. The Palestine Liberation Organization (PLO) was an Arab nationalist movement that formed in 1968 after the 6-day war (see below). The PLO defined Palestinian as “those Arab nationals who, until 1947, normally resided in Palestine regardless of whether they were evicted from it or stayed there. Anyone born, after that date, of a Palestinian father—whether in Palestine or outside it—is also a Palestinian. The Jews who had normally resided in Palestine until the beginning of the Zionist invasion will be considered Palestinians.”

Widespread use of the term “Palestinian” to refer exclusively to Palestinian Arabs is thought to have its roots in Soviet cold war propaganda. At the start of the cold war, the Soviet Union sought to build relationships in the Middle East. When Israel aligned with the US and the West, the Soviet’s launched an extensive anti-Israel propaganda campaign in the late 1960s. Many current criticisms of Israel spring from that campaign.

Genetics: The earliest recorded people of the Levant were the Canaanites. Most recent genetic studies show that both Jews and Palestinian Arabs share a great deal of common Canaanite ancestry and that Jews and Palestinian Arabs are more closely related to each other genetically than to other groups (see this 2000 study published in Human Genetics). This suggests that both groups have legitimate claims to being indigenous to the region despite both groups also having significant genetic admixture.

Origins of the Modern Conflict

Due to millennia of persecution by both Christians and Muslims, Jews had emigrated to and sought refuge in countries throughout Europe and the Middle East. In 1850, more than 27% of the population of Baghdad was Jewish (0% today). In 1931, 10% of Poland’s population was Jewish (0.01% today). For comparison, today, Jews comprise 16% of the New York City population and 2.4% of the US population.

Jews experienced regular pogroms (massacres and ethnic cleansing) and increasingly severe persecution throughout the 19th century. As Jews tried to flee, most countries, including the United States, closed their borders to Jewish refugees. An Austro-Hungarian Jew, Theodor Herzl, founded a Jewish nationalist movement called Zionism to rebuild the Jewish homeland in Israel and provide Jews with a safe place of refuge. Zion literally refers to a hill in Jerusalem but throughout Jewish history has symbolized Israel itself. The combination of severe persecution and no place to take refuge made Zionism critical to diaspora Jews.

During the 19th century, Palestine was under Ottoman Turkish rule and sparsely populated with roughly 300K residents, 85% of who were Muslim. European Jews started to purchase land in Palestine from Turkish and Arab land owners. As global persecution increased, many Jews started to move to Palestine in what diaspora Jews refer to as Aliyah. The influx of immigrants often resulted in friction between Jewish and Arab communities.

An Arab nationalist movement also began in the late 19th century, seeking to overthrow European colonial rulers. Both Jews and Arabs sought independence and autonomy and where these two nationalist movements overlapped (in Palestine), is the origin of the Arab-Israeli conflict.

World War I and the Balfour Declaration

Toward the end of World War I, in 1917, British Jews lobbied for a Jewish homeland and the British government announced their support for “a national home for the Jewish People” in Palestine in the Balfour Declaration. The specifics, including borders and whether that national home would be a state were vague. The declaration included calls to safeguard the civil and religious rights of the people living in Palestine, however the declaration itself was made without involvement of those people which infuriated Arab nationalists.

When World War I ended, the Ottoman empire had been defeated and was divided among the European powers. Nearly the entire Middle East was under European rule. A large area called Trans-Jordan fell under British rule and included British Mandatory Palestine.

Important Context: The period between World War I and World War II marked the beginning of the end of European colonialism throughout much of the Middle East. The European colonial powers and the League of Nations worked to define new independent nation states, often along ethnic lines. However, European interests were always kept in mind and national boundaries were often drawn to give power to favored groups or rulers. For example, the land on which the 35 million Kurdish people have always lived was carved up into 4 nation states (none Kurdish).

The issues of identity and perception are central to understanding the conflict: Arabs viewed diaspora Jews as Russian or Polish: European colonialists. Russians and Poles viewed Jews as an entirely different race, often killing or expelling them. Jews viewed themselves as an indigenous people returning to and reclaiming their homeland.

Peel Commission

With increasing Jewish immigration, conflict between Arabs and Jews increased. Having access to modern European agricultural techniques and education, Jewish immigrants dramatically improved the land they had purchased, converting sand dunes and swamps into orange groves. The Arab population grew quickly, but so did the economic disparities. As conflict and nationalism grew, both groups fought with each other and with the British, and the British sought a solution.

In 1936, the British Lord Peel was appointed to investigate and resolve the conflict in Mandatory Palestine. Peel recommended partition of the land into two separate states: one for the Arabs and one for the Jews and that a fusion was not possible. This proposal was unanimously rejected by the Arabs who opposed any Jewish state. The Zionists accepted the concept of partition but debated the specifics. At the Bloudan Conference in 1937, the entire Arab world rejected partition or any Jewish state and claimed the entire region as Arab.

Important Context: Arab rejection of any Jewish state has been central to the intractability of the conflict. For cultural, demographic, and religious reasons, Arabs sought absolute dominion over the entire region. Jews, having experienced severe persecution under Arab and European rule, will not accept a solution that does not provide autonomy and self-determination for Jews.

World War II and the Holocaust

World War II broke out in 1939 and the German Nazis launched the worst genocide in history, murdering 6 million Jews, nearly 2/3 of the entire European Jewish population. Jews who tried to flee found no sanctuary; all countries closed their borders to them, including the United States. Jews were rounded up into concentration camps and killed en-masse. The genocide is known today as the Holocaust.

Most Arabs aligned with the Nazis who were fighting the other European colonial powers that had occupied the Middle East. The Grand Mufti of Jerusalem, Muhammad Amin al-Husayni, was the Arab leader in Palestine and was famously antisemitic. He allied with the Nazis and supported the Jewish genocide. He also imported many Nazi ideas about Jews and European antisemitism to the region – ideas that had not been prevalent in the Islamic world previously. This greatly exacerbated antisemitism in the Arab world.

At end of the war, Europe had millions of displaced persons (DPs). Most returned to their countries of origin…except the Jews. The countries where they had previously lived would not accept Jews back and often killed them if they returned. So nearly a million Jews remained as DPs in the former concentration camps until modern Israel was founded.

Modern Israel

To address the hundreds of thousands of displaced Jews languishing in the “liberated” concentration camps, and to resolve the conflict between Arabs and Jews in Palestine, in 1947, the newly formed United Nations proposed the United Nations Partition Plan for Palestine. This plan (Resolution 181) called for the creation of separate Arab and Jewish states in British Mandatory Palestine with Jerusalem falling under international rule. Jewish organizations accepted the plan; the Arab League saw the partition as unfair and viewed any Jewish state as unacceptable, and rejected the proposed partition.

In 1948, the Jews declared independence in the portion of Palestine allocated to them in the UN partition plan and the next day, the surrounding Arab nations attacked with the intent of claiming all of Palestine for Arab rule. This was the first of three major Arab-Israeli wars.

1948 Arab-Israeli War

Israel declared independence on May 14, 1948. Much of the local Arab population and the surrounding Arab states of Egypt, Trans-Jordan, and Syria attacked the next day. After a 9 month war, the Jews had survived and modern Israel was established. Jordan conquered and annexed the part of Palestine west of the Jordan River (what we now refer to as the West Bank); Egypt conquered and occupied the part of Palestine we now refer to as the Gaza strip. Jordan held East Jerusalem and Israel held West Jerusalem.

Mass Displacements

The 1948 war resulted in the displacement of roughly 700K Palestinian Arabs who fled or were expelled from their homes in the area that became Israel. Palestinian Arabs refer to this, and to the formation of Israel generally, as the Nakba (meaning “catastrophe”). The Arabs who fled or were driven out had their property confiscated. Many of their descendants remain stateless and in refugee camps today. The Arabs who had remained in the newly formed Israeli state retained their property and were granted full citizenship. Today, more than 21% of Israel’s citizens are Arabs. Israeli Arabs have full citizenship and equal rights with Israeli Jews.

All Jews were ethnically cleansed from the areas conquered by the Arab states. Jordan expelled or killed all Jews from East Jerusalem, confiscated the land they had purchased during the Ottoman empire, and bulldozed ancient Jewish holy sites. Jordan refused access to Jerusalem to all Jews.

Moreover, roughly 900K Jews were then expelled from the rest of the Muslim world. Nearly 100% of the Jewish residents of the middle east were ethnically cleansed and their property confiscated. Where Baghdad had been 27% Jewish, only 26 Jews remain in all of Iraq today.

Important context: The conflict between the Arab and Jewish peoples has always been larger than Palestine. When Israel was formed, Jews were not just expelled from those areas of Palestine conquered by Jordan and Egypt, they were expelled from the entire Arab/Muslim. Jews who had no relation to Palestine and had lived in other parts of the Middle East for centuries. This is important because it explains why Jews will not accept a solution to the conflict that denies them a Jewish state.

Israel – a country of refugees

The majority of Israeli Jews today are people who have always lived in the Middle East (known as Mizrahi Jews) or their descendants and were made refugees by the mass displacements mentioned above. Another large group of Israeli Jews were the hundreds of thousands of displaced persons (DPs) who arrived as refugees from Europe. After World War II, millions who had been displaced languished in DP camps. Jews were among the last who could not be repatriated because so few countries would accept them (including the United States). Israel was the only country that would take the Jewish DPs.

Important context: Israel is a country of refugees. Arab nationalists have tried to apply the strategies used to drive European colonial settlers to return to their countries of origin from places like Algeria. That strategy has not worked with Israeli Jews because they are refugees, not colonists; they have no country to return to. Israeli prime minister, Golda Meir, famously quipped: “We Jews have a secret weapon in our struggle with the Arabs; we have no place to go.”

Palestinian Arab refugees

While Jewish refugees were absorbed into Israel in the years following its formation, the surrounding Arab states did not accept the displaced Palestinian Arabs as citizens and kept them in refugee camps along the Israeli border, believing that they would soon conquer Israel and return the refugees to their homes. This created the Palestinian refugee problem that persists to this day.

Important context: Arab League nations are dictatorships, monarchies, and theocracies. Ethnic nationalism is an effective tool for distracting populations from economic and political problems. Palestinian Arab refugees have been incredibly useful in this regard and this is part of the reason so many remain stateless and have not been offered citizenship in surrounding Arab countries that share language, cultural, religious, and ethnic ties.

UNRWA

Refugees from all global conflicts except in Palestine are managed by UNHCR whose mission is to help displaced people find new homes and citizenship (at which point they are no longer refugees). However, In 1949, the UN formed a new agency exclusively to serve Palestinian Arab refugees: the United Nations Relief and Works Agency (UNRWA).

Unlike UNHCR, UNRWA does not seek to resettle refugees and counts as refugees people who have moved to other countries and become citizens there (something done in no other conflict) – and their descendants! As a result, UNRWA now provides assistance to 1.5 million Palestinian Arabs and has registered 5.6 million as refugees.

UNRWA is frequently criticized by Israel and the United States for being beholden to terrorist organizations such as Hamas and Islamic Jihad.

Important context: UNRWA is viewed as controversial because its structure and mission effectively perpetuate the conflict. Under UNRWA, the 700K initial refugees have grown to 5.6 million. UNRWA has 30K staff members (almost all Palestinian) while UNHCR, which serves the entire rest of the world, has a staff of 18.9K.

1964 Palestinian Liberation Organization (PLO) and Terrorism

In 1964, at a meeting of the Arab League, a coalition entity called the Palestine Liberation Organization (PLO) was formed and was closely associated with the Pan-Arabist movement. Among the goals stated in the PLO charter were the complete destruction of Israel. Note that the PLO was formed 3 years before the 1967 war – a time when the West Bank was controlled by Jordan and the Gaza Strip was controlled by Egypt. The goal of the PLO was to destroy Israel entirely.

Following the 1967 war (see below), Yasser Arafat became chairman of the PLO. The PLO was initially based in Jordan where it waged guerilla war with Israel, but after trying to overthrow the Jordanian monarchy, the PLO was expelled and went to Lebanon. After sowing discord in Lebanon leading to civil war, and eventually drawing Lebanon into a catastrophic war with Israel, the PLO was also expelled from Lebanon and scattered to various sympathetic Arab countries.

During the 1970s, the PLO shifted from a strategy of guerilla warfare against Israel to international terrorism and a focus on civilian targets. Palestinians hijacked airliners, cruise ships, restaurants, airports, and the Olympics.

Important context: One of the PLO’s terrorist acts was the hijacking of a French civilian airliner, taking hundreds hostage. The hijackers flew the plane to Uganda where they received support from dictator Idi Amin. After releasing all non-Jewish hostages, the hijackers threatened to kill the remaining 102 Jewish passengers. Israel launched a daring midnight raid on the Entebbe airport with 100 commandos, freeing all but 3 of the hostages and killing the hijackers and dozens of Ugandan troops. Only one commando was killed: Yonatan Netanyahu – the older brother of Israeli prime minister Benjamin Netanyahu. Also important: the PLO terrorism was not constrained to Israelis; terrorist would separate anyone who had a “Jewish sounding” name and viewed all Jews globally as targets.

1967 Arab-Israeli War (aka six-day war)

In 1967, Arab states (mainly Egypt) massed their armies on the Israeli border and Egypt closed shipping lanes to Israeli vessels. Egypt ordered withdrawal of UN peacekeeping forces (who interestingly left without comment). Rather than wait for the invasion, Israel launched a series of preemptive airstrikes and destroyed the air forces of the attacking countries (Egypt, Iraq, Syria, Jordan). This left Israel with air supremacy; they then launched a series of quick ground offensives which routed the Arab forces. The Egyptian forces were defeated and Egypt lost the Gaza strip and the entire Sinai peninsula. Jordan lost the West Bank, including West Jerusalem, and the Syrians lost the Golan Heights.

Critically, by the end of the war, nearly 1 million Arabs now lived in territory occupied by Israel but not annexed by Israel: the West Bank and the Gaza Strip. This greatly exacerbated the refugee crisis.

1970 Black September (PLO in Jordan)

Palestinian guerilla forces (the PLO and related factions) began operating out of Jordan in the aftermath of the 1967 war. Jordan had been formed from the majority of the British post WWI territory and had a majority Palestinian population. Jordan’s King Hussein allowed the PLO to launch attacks against Israel and engage in international terrorism such as the 1972 Olymbic Munich Massacre and numerous hijackings of international airliners. As the PLO became more powerful, they sought to seize power in September 1970 and a bloody civil war ensued. Eventually, Jordanian forces routed the PLO who fled to Lebanon (where they later became pivotal in the Lebanese civil war).

1973 Arab-Israeli War (aka Yom Kippur war)

In 1973, a coalition of Arab states launched a surprise attack against Israel on Yom Kippur (the holiest day in the Jewish calendar). Although initially caught off guard, Israeli forces prevailed, pushing the Syrian army back to Damascus and coming within 100km of Cairo. The war was particularly dangerous globally because by this time, the United States was firmly allied with Israel and the Soviet Union with the Arab states. This was a the height of the cold war and the risk of escalation between the super powers was serious.

Israel and Egypt later negotiated a peace treaty wherein Israel returned the Sinai peninsula and Egypt recognized Israel and eventually moved out of the Soviet sphere of influence into the American sphere.

1975 Lebanon Civil War (PLO in Lebanon)

Lebanon was already in trouble when the PLO arrived from Jordan, but their arrival helped trigger the start of a decade of civil war that effectively destroyed the country. The PLO presence in Lebanon lasted until 1982 when Israel finally drove them out, but in the interim, they gained near full control of southern Lebanon and continued to launch attacks on Israel and engage in international terrorism. They were a major player in the brutal multi-party Lebanese civil war that left the country in ruins.

1978 Camp David Accords

One of the first moves towards peace between the Arabs and the Jews took place in 1978. US president Jimmy Carter hosted Egyptian president Anwar al-Sadat and Israeli prime minister Menachem Begin for secret negotiations at Camp David in Maryland, resulting in the Camp David Accords. In exchange for peace and recognition, Israel returned the Sinai peninsula to Egypt. Israel and Egypt agreed to a framework for peace for the Palestinians (but the Palestinians did not participate in the negotiation).

Sadat and Begin received the 1978 Nobel peace prize. Egypt and Israel normalized relations in 1980, ending their state of war. Israel dismantled Israeli settlements from the Sinai and removed all settlers. Egypt was suspended from the Arab League for ten years. Sadat was assassinated in 1981 by members of the Islamic Jihad.

1982 Lebanon War

Following their exile from Jordan, the PLO set up base in southern Lebanon from which it regularly attacked northern Israel. Lebanon had been in the throes of a sectarian civil war since at least 1975, with Israel allied with the Druze/Maronite Christians and Syria backing the PLO forces. An attempt to assassinate an Israeli ambassador was the incident that prompted Israel to invade southern Lebanon to dislodge the PLO. The Israeli army defeated the Syrian and PLO forces who were surrounded. American intervention negotiated an evacuation of the PLO from Lebanon to Tripoli. The PLO military wing was permanently diminished. The Syrian proxy was replaced with a pro-Israeli Christian government, but the president was soon assassinated and the civil war in Lebanon continued. The devastation from the war with Israel left Lebanon more cautious about future war with Israel.

Important Context: The devastating civil war in Lebanon had little to do with Israel and is indicative of the larger geopolitical context in which Middle East conflict occurs. The battles in Lebanon were sectarian, pitting Christians, Sunni Muslims, an Shia Muslims against each other. These battles were often fought by militias that were proxies for Syria, Iran, Iraq, and others. Lebanon ultimately ended up under Syrian/Iranian control. Today, the Iranian-proxy militia Hezbollah controls much of Lebanon.

1987 Intifada

Israel had occupied the West Bank and Gaza strip since 1967, but had not annexed them, instead starting a prolonged period of military occupation. The occupation left a generation of Arab residents stateless and in political limbo. Moreover, when Jordan had captured the West Bank in 1948, all Jews had been killed or expelled and their property seized. Certain denominations of religious Jews believed that Israel should return to and annex the West Bank, which had been part of ancient Israel (the kingdoms of Judea and Samaria). They began building new Jewish villages there. These became known as “settlers” and have become a flashpoint for conflict ever since.

Frustrated by living under the brutality of military occupation, with the PLO greatly diminished since being exiled from Jordan and Lebanon, and with Jews moving back into the West Bank the first Intifada started. This intifada was characterized by civil disobedience, property damage, and periodic violence using mainly stones and molotov cocktails (petrol bombs). Israeli forces responded with a heavy-handed crackdown that fed Arab resentment and further inflamed tensions.

In the aftermath of the first Intifada, the PLO pivoted toward considering a two-state solution, but competing groups such as Hamas sprung up dedicated to the destruction of Israel and genocide of all Jews.

1987 Hamas

Hamas emerged in 1987 as a competitor to the PLO. A branch of the 1928 Egyptian Muslim Brotherhood, and active in the Gaza strip since the 1950s, their 1988 Charter called for the destruction of Israel, killing of all Jews, and establishment of a fundamentalist Islamic state in all of Palestine. Where the PLO was nationalistic, Hamas is a fundamentalist religious organization whose charter explicitly calls for genocidal violence by divine decree. Hamas is a designated foreign terrorist organization by the US government (and most other governments).

Important Context: Hamas is ideologically committed to destroying Israel and killing Jews. Although some efforts were made to soften those positions in 2017, they are widely regarded as propaganda; the 1988 Charter was never changed or rescinded. Hamas does not want peace with Israel or a two-state solution. Hamas’ belief in the divine nature of their mission has allowed them to engage in the most brutal forms of violence without remorse.

Important Context: Muslim Brotherhood is a larger organization with a mission of global Jihad. MB has played a critical role in uprisings in Egypt and today is backed primarily by Qatar and Turkey. The conflict in Israel/Palestine is part of much larger regional power struggles.

1993 Oslo Accords

Although the first Intifada failed, it changed the status quo for Israelis and Arabs. As Israel had grown in military and economic strength, it had gradually become apparent that Israel could not be defeated militarily by the PLO or surrounding Arab states. Normalization with Egypt had proven resilient and Jordan was done with the PLO after Black September. Israel, for the first time, seemed to be in a position where it no longer faced immediate existential threat.

However the Intifada demonstrated to Israelis that even overwhelming military strength could not guarantee Israeli security against domestic uprisings. Moreover, many in Israel felt the decades-long occupation was unjust and could not continue indefinitely.

Following a conference in 1991, Israeli PM Rabin and PLO head Arafat met with US President Clinton and signed the Olso I Accord wherein the PLO renounced violence and their goal of destroying Israel and Israel committed to a path for Palestinian autonomy and withdrawal from the Gaza Strip and parts of the West Bank over the next 5 years. The PLO recognized Israel and Israel recognized the PLO as the representative of the Palestinian people. Numerous issues remained unresolved and their “final status” was to be negotiated over the 5 year period. Rabin, Perez, and Arafat received Nobel peace prizes in 1994. Additional agreements were signed in 1995: the Oslo II Accord that set out the basis for further negotiations.

1994 Palestinian Authority

As part of the Oslo Accords, Israel withdrew its forces from parts of the West Bank and a new Palestinian government was established: the Palestinian National Authority or just Palestinian Authority or PA. This became the de-facto government of the Palestinians in parts of the West Bank and all of the Gaza strip.

Israeli settlement activity had been ongoing since 1967 and there were now large Israeli towns (aka settlements) in the West Bank. For this reason, Oslo divided the West Bank into 3 regions: A=Palestinian urban areas, B=Palestinian rural areas, C=Israeli settlements. The PA had full civilian and military control over area A and full civilian control over area B. Israel had full civilian and military control over area C and military control over area B. In 1993 there were roughly 111K Israelis living in area C.

The PA was domestic; the PLO continued to represent Palestinians internationally including at the UN.

2000 Camp David Summit and the Second Intifada

In July of 2000, president Bill Clinton facilitated peace negotiations between Israeli prime minister Ehud Barak and PA chairman Yasser Arafat. This meeting, known as the 2000 Camp David Summit, lasted two weeks and was comprised of largely oral proposals between the two parties. Talks reached an impasse at many issues, but one of the primary challenges was that Israelis refused the “Right of Return” (more below) and Arafat refused to compromise without this, rejecting the Israeli offer to establish a demilitarized Palestinian state with 100% of Gaza and ~90% of the West Bank. Another major issue was that corruption and violence within the PA was pervasive, and a belief among many Israelis that the PLO was not sincere in its recognition of Israel and would merely use any new state as a launching pad for terrorism and war. Trust was low and faltered. The summit concluded without an agreement, which ignited a 5-years long Palestinian revolt, known as the Second Intifada, involving riots, lynchings, rocket launches, and suicide bombings.

The final status issues of the summit were thorny, with central challenges including:

  • “Right of Return”
    Palestinian Arabs demand for a “right of return” for Arabs displaced by the 1948 war *and their descendants*: a right to return to their ancestral homes in Israel and the West Bank. This too is something Israelis will almost certainly never accept for two reasons: 1) High Palestinian birth rates have resulted in a soaring population of descendants who could potentially become the majority in Israel if allowed to return freely. Israel would be faced with the prospect of Israeli no longer being Jewish or no longer being a democracy; both unacceptable to Israeli Jews. Moreover, many Israelis view the “right of return” as fundamentally unfair since in 1948, similar numbers of Jews were ethnically cleansed by Arab states and neither they nor their descendants will ever regain their homes and property.
  • Final status of Jerusalem
    Jerusalem is the the capitol and the spiritual home of the Jews. For thousands of years, Jews have ended prayers with “Next year in Jerusalem”. Jerusalem is where the holiest Jewish sites are including the Temple of Solomon which was destroyed, rebuilt, destroyed again, and then Arabs built a mosque on top of the site (the Dome of the Rock) which is now the third holiest site in Islam. When Jordan captured East Jerusalem, they expelled all Jews and confiscated their property. Jewish holy sites were desecrated and Jews lost all access to them. Palestinians now seek East Jerusalem (or alternatively all of Jerusalem) as the capitol of their new state. This is something Jews will almost certainly never accept.
  • Settlers
    The settler movement had continued and was making it increasingly difficult for a final Palestinian state to be contiguous rather than a patchwork of disconnected areas. By 2000, there were nearly 200K Israeli settlers living in the West Bank; Barak proposed to reabsorb these settlers by moving them to the portion of the West Bank which Israel sought to permanently annex.

Terminology and Symbols: The suicide bombings and other violent attacks seen in the Second Intifada were typically aimed at Israeli civilians, claiming hundreds of lives across dozens of terrorist attacks. Consequently, Israelis and the Jewish diaspora associate the term “intifada” with indiscriminate violence against civilians and the Jewish people. Another symbol to arise from this violence was that of bloodstained hands, as seen in the infamous photo of the 2000 Ramallah lynching.

2001 Taba Summit

The second intifada led to unrest among Israeli Arabs, resulting in protests that devolved into violent clashes with the police. Barak came under political fire for this turmoil and resigned in late December 2000. Barak and Arafat continued peace talks for a week at the Taba Summit in late January of 2001; these talks remain the closest to peace the two sides have come. The final status issues still largely revolved around the “Right of Return” and details of Israeli annexation of pieces of the West Bank. Both sides agreed to Jerusalem being an open city and having separate Arab and Jewish neighborhoods with Palestinian and Israeli sovereignty, respectively. Despite the productive nature of the talks, the talks tragically ended about a month prior to an agreement could be reached; Ariel Sharon was elected prime minister with a 2:1 lead, and he did not resume negotiations after his election.

Important Context: although the Likud party did not resume peace negotiations after the Taba Summit, it is important to consider that such talks were occurring under the backdrop of the second intifada. Tension between the Israeli Jewish and Arab citizens and distrust between the PA and Israel remained high, which helped undermine efforts for peace and bolstered support for right-wing politics.


2005 Gaza Withdrawal

2007 Hamas

2020 Abraham Accords

Prospects/Obstacles to Peace

Proxmox VMs

It takes a lot of time to setup a server and then it must be maintained including regular backups. Virtualization can help with a lot of this. Modern computers have lots of cores, memory, and disk space so it is now possible to run multiple servers as virtual machines within a single physical server. This arrangement offers lots of advantages including:

  • Use resources efficiently – many servers only use a small fraction of the physical machines capability so you can easily run quite a few virtual servers on one physical machine.
  • Keeping servers and their environments separate helps avoid conflicts
  • Easily perform “bare metal” backups of virtual servers and restore them to the same or a new physical server for quick disaster recovery.
  • Easily allocate and expand resources (within the limits of the physical server)

I generally run my home servers on Intel NUC platforms because they offer a nice balance of computing power and power efficiency. A basic NUC 12 Pro with an i5-1240P or higher processor has at least 12 cores, up to 64GB of RAM, a fast NVMe gen 4 drive, 2.5GbE, and a TDP of only 28W. For bulk storage, you can use a NAS or connect a DAS via USB3.2 for very high speed. They stack, they’re small, quiet, and the low power consumption means a typical UPS will carry them through most outages. In short, they’re great little servers.

For virtualization, I like Proxmox. Proxmox is Debian based; it installs quickly from a USB flash drive and provides a friendly web-based management interface that is exactly what’s needed. It allows you to see the status and manage both the physical server and the VMs. It has a tightly integrated KVM hypervisor so you can access the console of each VM and the physical server remotely via the web interface.

Proxmox also makes it easy to make “bare-metal” backups which take a snapshot of the entire VM that can be easily restored in case of disaster either on the same physical server or a new server. The backup files are sparse and compressed; a machine with 64GB of storage that is using 24GB will yield a snapshot file ~12.5GB. You can download the snapshots and store them on bulk storage and off-site. The fact that Proxmox is so easy to install and that you can then restore entire VM snapshots quickly means that even if the physical server and/or storage failed completely, you can be back up and running on a new machine in less than 30 minutes. (I have done this)

Many servers don’t need a lot of compute power; most of mine do just fine with 4-8GB of RAM 2-4 cores, and 32-64GB of storage. This means I can host several servers on a NUC12 without it breaking a sweat. Keeping servers separate (e.g. database, middleware, web applications, etc.) makes it easy to scale and to upgrade individual servers without software or hardware conflict with others.

Update Sept 2024:
I have been using Proxmox for a few months now and still like it a lot; it simply works. The web-based management is perfect – exactly what’s needed for managing VMs with excellent status-at-a-glance and detailed configuration pages. I have resized VM hardware allocations on the fly several times and the interface is fast and intuitive. The bare-virtual-metal backups are easy to do and give me confidence about disaster recovery.

The only thing I think proxmox should change is their lowest tier (community) pricing/model:

  • I hate renting software; if they simply charged a fixed price with optional annual maintenance, most serious users (including me) would likely end up buying the maintenance anyway since nobody wants to run a server without the latest security fixes. However, I don’t want that decision forced on me; I like to own not rent.
  • The per-socket pricing effectively forces certain hardware choices: in particular, a single high-power server over a few low-power machines (like NUCs) that have been re-purposed. This removes a certain level of flexibility (the opposite of everything else about proxmox). For the community model, a site-license that covered a small number of machines or some aggregate level of computing power might make more sense.
  • Finally, $110/year/socket is well above the no-brainer cost for most users. I’d bet proxmox would net significantly more revenue if they lowered their community pricing to the ubiquitous $99 purchase + $49/year optional maintenance.