Should Your Web History Impact Your Credit Score? The IMF Thinks So
The rise of fintech services and cryptocurrencies have changed modern banking in a number of ways, and banks face an increasing number of challenges as various third-party payment processors interpose themselves between financial institutions and their traditional customers. The credit scoring systems used broadly in the US and Europe are based on so-called “hard” information — bill payments, pay stubs, and how much of your current credit limit you are tapping.
The researchers point out that so-called “hard” credit scores have two significant problems. First, banks tend to reduce credit availability during a downturn, which is when people most need help. Second, it can be difficult for companies and individuals without credit histories to begin creating one. There’s a bit of a catch-22 in the system, in that what you need to persuade an institution to loan you money is a credit history you don’t have because no one will loan you money.
Having identified two flaws in the existing system, the authors write:
The rise of the internet permits the use of new types of nonfinancial customer data, such as browsing histories and online shopping behavior of individuals, or customer ratings for online vendors.
The literature suggests that such non-financial data are valuable for financial decision making. Berg et al. (2019) show that easy-to-collect information such as the so-called “digital footprint” (email provider, mobile carrier, operating system, etc.) performs as well as traditional credit scores in assessing borrower risk. Moreover, there are complementarities between financial and non-financial data: combining credit scores and digital footprint further improves loan default predictions. Accordingly, the incorporation of non-financial data can lead to significant efficiency gains in financial intermediation.
In a blog post published on the IMF’s website, the authors also write: “Recent research documents that, once powered by artificial intelligence and machine learning, these alternative data sources are often superior than traditional credit assessment methods.”
However much the authors of this paper know about banking systems and finance, they’re clearly not up to date on the latest in AI research. This is a bad idea in general, but it’s a really terrible idea right now.
The first major problem with this proposal is there’s no evidence AI is capable of this task or that it will be any time soon. In an interview with The Guardian earlier this summer, Microsoft AI researcher Kate Crawford had some harsh remarks for the current reality of artificial intelligence, despite working for one of the leaders in the field: “AI is neither artificial nor intelligent. It is made from natural resources and it is people who are performing the tasks to make the systems appear autonomous.”
When asked about the specific problem of bias in AI, Crawford said:
Time and again, we see these systems producing errors – women offered less credit by credit-worthiness algorithms, black faces mislabelled – and the response has been: “We just need more data.” But I’ve tried to look at these deeper logics of classification and you start to see forms of discrimination, not just when systems are applied, but in how they are built and trained to see the world. Training datasets used for machine learning software that casually categorise people into just one of two genders; that label people according to their skin colour into one of five racial categories, and which attempt, based on how people look, to assign moral or ethical character. The idea that you can make these determinations based on appearance has a dark past and unfortunately the politics of classification has become baked into the substrates of AI.
This isn’t just the opinion of a single person. Gartner has previously projected that 85 percent of AI projects through 2022 “will deliver erroneous outcomes due to bias in data, algorithms or the teams responsible for managing them.” A recent Twitter Hackathon found proof that the website’s photo-cropping algorithm was implicitly biased against older people, disabled people, Black people, and Muslims, and it frequently cropped them out of photographs. Twitter has since discontinued using the algorithm because these kinds of bias problems are in no one’s best interest.
While my own research is far removed from fintech, I’ve spent the last 18 months experimenting with AI-powered upscaling tools, as regular wfoojjaec readers know. I’ve used Topaz Video Enhance AI a great deal and I’ve experimented with some other neural nets as well. While these tools are capable of delivering remarkable improvements, it’s a rare video that can simply be chucked into TVEAI with the expectation that gold will arrive out the other side.
Here’s frame 8829 from the Star Trek: Deep Space Nine Episode “Defiant.” The quality of the frame is reasonable given the starting point of the source, but we’ve got a glaring error in the face of Jadzia Dax. This is output from a single model and I blend the output of multiple models to improve DS9’s early seasons. In this case, every model I had tried was breaking in this scene in one way or another. I’m showing output from Artemis Medium Quality in this instance.
This specific distortion happens once in the entire episode. Most Topaz models (and every non-Topaz model I tested) had this problem and it proved resistant to repair. There aren’t very many pixels representing her face and the original MPEG-2 quality is low. There is no single AI model that treats an entire episode of S1 – S3 correctly that I’ve found yet, but this is by far the worst distortion in the entire episode. It’s also only on screen for a few seconds before she moves and the situation improves.
The best repair output I’ve managed looks like this, using TVEAI’s Proteus model:
There’s a reason why I’m using video editing to talk about problems in fintech: AI is nowhere near perfect yet, in any field of study. The “fix” above is imperfect, yet required hours of careful testing to achieve. Behind the scenes of what various companies smugly call “AI” are a lot of humans performing an awful lot of work. This doesn’t mean there isn’t real progress being made, but these systems are nowhere near as infallible as the hype cycle has made them out to be.
Right now, we’re at a point where applications can produce some amazing results, even to the point of making genuine scientific discoveries. Humans, however, are still deeply involved in every step of the process. Even then, there are mistakes. Fixing this particular mistake requires substituting output from an entirely different model for the duration of this scene. If I hadn’t been watching the episode carefully, I might have missed the problem altogether. AI has a similar problem in general. The companies that have struggled with bias in their AI networks had no intention of putting it there. It was created due to biases in the underlying data sets themselves. And the problem with those data sets is that if you don’t examine them with care, you might wind up thinking your output is composed entirely of frames like the below, as opposed to the damage scene above:
Even if the AI component of this equation was ready to rely on, privacy issues are another major concern. Companies may be experimenting with tracking various aspects of “soft” consumer behavior, but the idea of tying your credit score to your web history is very similar to the social credit score now assigned to every citizen by China. In that country, saying the wrong things or visiting the wrong websites can result in one’s family members being denied loans or access to certain social events. While the system contemplated is not that draconian, it’s still a step in the wrong direction.
The United States has none of the legal framework that would be required to deploy a credit monitoring system like this. Any bank or financial institution that wishes to use AI to make decisions regarding the creditworthiness of applicants based on their browser and shopping history needs to be regularly audited for bias against any group. The researchers that wrote this document for the IMF talk about hoovering up people’s shopping histories without considering that many people use the internet to shop for things they’re too embarrassed to walk into a store and buy. Who decides which stores and vendors count and which do not? Who watches over the data to make sure intensely embarrassing information is not leaked, either on purpose or by hackers more generally?
The fact that non-bank financial institutions may be jonesing to use some of this data (or already using it) is not a reason to allow it. It’s a reason to stay as far away from said organizations as possible. AI is not ready for this. Our privacy laws are not ready for this. The consistent messaging from reputable, sober researchers working in the field is that we’re nowhere near ready to turn such vital considerations over to a black box. The authors who wrote this paper may be absolute wizards of banking, but their optimism about the near-term state of AI networks is misplaced.
Few things are more important in modern life than one’s credit and financial history, and that’s reason enough to move exceptionally slowly where AI is concerned. Give it a decade or two and check back then, or we’ll spend the next few decades cleaning up injustices inflicted against various individuals literally through no fault of their own.
Continue reading
SpaceX Launches ‘Better Than Nothing’ Starlink Beta
Those lucky few who have gotten invitations to try the service will have to pay a hefty up-front cost, and the speeds aren't amazing. Still, it's a new generation of satellite internet.
PS5, Xbox Series X Thin on the Ground, Along with Everything Else
Microsoft and Sony are shipping every piece of hardware they can, but that doesn't seem to be very many consoles. Don't worry, though — everything else is hard to find, too.
PS5, Xbox Series X Thin on the Ground, Along With CPUs, GPUs
Microsoft and Sony are shipping every piece of hardware they can, but that doesn't seem to be very many consoles. Don't worry, though — everything else is hard to find, too.
Google Will Use Pixel’s Camera to Measure Heart Rate and Breathing
Like many of Google's machine learning projects, this one is coming first to Pixel phones, and more phones will probably get it down the line.