Any idea why they have suddenly been deemed as unfit to use?
HID and cheaper retro fit LED's bulbs are mostly designed for projector lenses, some will (quality units) work fine in a reflector headlight more on that later. Most won't as the arch globe, chip position, shape/length and how the light is made differ from a halogen filament and focal point of a reflector is matched to the filament position. Which means you get a crap light pattern and/or dazzle on coming traffic or both. The light produced by HID and now quality LED's is far superior can and will blind you, long enough to cause potential collisions. Interestingly you see quite a few cars of late with halogen bulbs, where one or both low beam/s dazzling, this seems to have coincided with the likes of Halfords bulbs fitting service. Fitting by poorly trained shop staff, who either fit the bulb upside down or not fully home, which moves the filament position and the focal point and you get the above. Some cars you now have to remove the headlight unit and/or bodywork, jack up or get access from underneath and even removal of a front wheel to change a bulb. Some try all manner of get arounds, which can end up with incorrectly fitted bulb as described above, due to inadequate access or incorrectly refitting of the above mentioned. I've been asked to correct a fair few over the years!
In brief the MOT used to look for beam pattern and height etc and there was/is points about self levelling and lens washing for car HID systems. But as technology has moved on these we're no longer needed and they sort of got ignored. But the issue above didn't go away i.e. dazzle and poor driving lighting due to cheap poor quality HID/LEDs, so now legislation has caught up a bit, effectively banning retro fit HID/LED's into reflector units not designed to run them.
Thing is, technology has moved on already
You can now and have been able to get for some years HID bulbs that are compatible with reflectors, that said HID is really obsolete. Quality LEDs now contain lighting chips which are pretty much identical in shape/size/position as halogen filaments and adjustable (rotation) so you can match the bulb to the reflector. i.e. in some units the bulbs is fitted so that the light is produced to the left/right/up/down, rotation means you can match that.