What is iBorderCtrl?
"The Minority Report" – coming soon to a border around you
iBorderCtrl is an EU-funded automated border security system currently being tested. Supposedly, it will make crossing non-EU land borders into the EU more efficient. As part of iBorderCtrl, travelers who do not have EU passports have to sign up in advance of their trips, just like you have to in some other countries today, like when traveling to the United States as a non-citizen.
In the iBorderCtrl developers' vision, signing up to travel to the EU would not just involve giving the system your name and address, but all sorts of details including information about your social media accounts. The system will then ask you to turn on your webcam so that a humanoid avatar on your screen can interrogate you while the “Automatic Deception Detection System” Artificial Intelligence running in some server rack far away studies your face for minuscule motions that supposedly betray that you are lying.
And that's only before you even get to the border. Once there, the border guards will have all sorts of biometric devices for you and a 21st-century-DDR “Hidden Human Detection System” for your car.
But you have nothing to worry about. After all, you talk to your Twitter followers like you would to open-minded border guards in fine democracies like Hungary. No reason to be nervous when the AI questions you.
Welcome to Europe! Feel like you're living in the future yet?
Wait... Wait... What?
Before we get into why all of this is even more problematic than you might think, let's recap iBorderCtrl in its creators' own words.
a) Pre-Travel (First Stage)
The Pre-Travel Stage is designed to gather information […] through the Traveller User Application (TUA). […] enter and update their personal information, upload travel-related documents (such as VISA, passport), travel information (hotel reservation, vehicles data, etc.) and undertake an avatar interview. […] Following this procedure, the traveller is requested:
TUA will verify that the traveller has entered all the mandatory information, check the authenticity of the uploaded travel documents and will store all information in the iBorderCtrl database. As a next step, the traveller will go through an interview with a virtual border agent (avatar) conducted by the Automatic Deception Detection System (ADDS). The avatar is an artificial figure which represents a border guard. During the avatar interview, a set of traveller and travel related questions will be asked. Such questions are the same questions an actual border guard may ask travellers in a real-life border crossing scenario. The avatar interview is designed to detect false answers as it observes non-verbal behaviour. This means that the traveller will be filmed using his/her video camera while computer software observes facial (micro) gestures of the participant to detect deceptive behaviour. In the following step, a face matching will be conducted in which the traveller’s passport photo will be compared to a short video sequence of the traveller that was gathered during the avatar interview.
Please note that the pre-travel registration intends to involve checks against some law enforcement databases such as the VIS, SIS II etc., as well as publicly available information of the traveller from social media (twitter) to verify the information he/she provided as well as whether the traveller poses any public security risks. […] Based on such database entries, the documents, the traveller has provided, the answers he/she has given in the avatar interview, the outcome of the face matching procedure and his/her gestures that occurred during the interview a risk classification for the particular travel will be calculated by the Risk Assessment Software Tool (RBAT). This risk assessment result will not be released to the traveller. […]
b) Border Crossing (Second Stage)
[…] Travellers must present their QR codes and the border guard will, by scanning the QR code via a portable unit, access traveller’s personal information gathered during pre-travel registration, as well as the risk assessment score. The border guard will then validate the authenticity of the travel documents using a portable scanning device. After that, he will validate vehicle information (e.g. registration, plate number etc.) and proceed to match the fingerprint reference with the traveller. […] Further biometric validation will be performed if the fingerprints don’t match or if travellers’ behaviour and/or information provided has led the border guard to suspect that they have given false information about their identity. This includes the use of facial recognition and palm vein technologies. Through the facial recognition technology, the facial image of the traveller will be captured using a camera on the portable unit. The video sequence of the face image will then be compared with the photo stored on the traveller’s passport or visa and with the short video sequence of the traveller that was gathered during the avatar interview at the pre-registration phase. […] Border check would then move on to the hidden human detection (applicable only in the case where the traveller crosses the borders using his/her private car). All the information gathered will be analysed to provide an overall risk level to assist the border guard in deciding about the individual traveller.
The above text is an excerpt from their “Informed Consent” document, because the present test is completely voluntary. Because after all, who in their right mind wouldn't want to volunteer as a test subject for technology that can tell if you're lying? (Hey, if you don't like it, they'll even remove your data. Just send an email to their privacy officer, who just happens to be a Hungarian Police Major with a Gmail address. We wish we were kidding.)
So what is wrong with iBorderCtrl?
Our most fundamental issue is with lie detection being incredibly intrusive. It attempts (currently rather badly, we'll come back to that) to get at what is inside your head. Even the author of arguably the most famous (and sadly also visionary) dystopian novel did not imagine the rulers probing the mental state of the inhabitants of Airstrip One:
Always the eyes watching you and the voice enveloping you. Asleep or awake, working or eating, indoors or out of doors, in the bath or in bed—no escape. Nothing was your own except the few cubic centimetres inside your skull.
– George Orwell in 1984
We would argue that there is a right to “cognitive liberty”, and that the inner workings of people's brains are off-limits to the state.
There are other fundamental issues, such as whether we want to automate away the humans from interactions with law enforcement. There is a reason that when we call the police, we expect human beings to show up, and not robots. Suppose the robots were “just there to help human officers make a decision later”; would that be ok with you?
And then there's the creepy social media stuff. Picture that in an offline world. “Before we decide, let's see whether you have written any op-eds in the past years…” And do you think they'll stop at Twitter? Don't you think they'll want to see what you talk about in a more private setting? Or do you think Mark Zuckerberg is going to stop them?
Lie detection is pseudoscientific
You may have seen lie detectors in American movies or on tawdry talk shows. And if you seem to remember that they are somehow discredited technology, you would be absolutely right. There is so much to say here that we wrote a longer text for those who want to know more about polygraphs (as the Americans call lie detectors), what they are doing for iBorderCtrl, and why it has already been repeatedly and widely agreed that all of this lacks sufficient scientific basis for mass screenings of any kind.
The tl;dr is that what is being described as “lie detection” does not actually exist, and what they are doing for iBorderCtrl does not qualify as proper science. And “lie detection” using AI to classify internal states has a number of additional problems. To name just a few, there's differences between racial and ethnic groups, there's the existing bias of whatever “ground truth” you train the AI with, and then there's the little fact that the AI cannot explain why it has decided the way it did.
Don't think this will stay where it is being tested right now. They're using this on non-Europeans because that is what they can currently get away with. But if they succeed here, we can look forward to a future where our every minuscule facial motion is being interpreted by questionable AI systems that claim to be able to look into our heads. Remember that just because it's probably junk science doesn't mean it can't ruin your whole day, especially if people in positions of power trust their shiny new machines.
We're in the dark
One of the more disconcerting things about the current “lie detection” technology being rolled out is how precious little we know. All the studies cited on the iBorderCtrl website have been done by a few scientists. These same scientists also form the team from Manchester Metropolitan University that is a partner in the EU-funded consortium. The technology for the experiment comes from a startup company owned and operated by… the same scientists. Confidential progress reports to the EU have been written by… the same scientists. But at least the technology was tested before being brought to the border, by… the same scientists (on 30 of their own employees). The data from those tests is just as inaccessible as the data from the current wider experiments at the borders will be: It will only ever be seen by the companies in the iBorderCtrl consortium. And by the Hungarian police, that goes without saying.
Despite this complete lack of information, the one repeated take-away that we get from their publicity efforts is that this is a “success story.” High accuracy claims, questionable research methods, no publicly verifiable data, and a lot of money to be made. What could possibly go wrong?
Depressed?
All is not lost. There is plenty we can do, especially if more than a few of us get more than a little upset.