Sunday, June 16, 2013

Download your note list

I have added a new feature to allow you to download a listing of your notes through the Prosper API.  You can then use these lists of notes on the Loan Analysis or the Previously Late Loans tools to help manage your portfolio.

Currently the download feature is only capturing newly originated loans.  Loans bought/sold on Folio are not being taken into account.  This mimics how the tools used to work when looking at the previously public bid data.


To add notes:


Once you sign in to Prosper-Stats, your email address will display on the top left of the page next to the Sign Out link.  You can click your email address to go to your profile page.  This page will show you information on any note lists you have downloaded and provide you a link to download notes from Prosper.

You will need to enter your username and the password for the API (which should be different then the password you log in to the website with) to download notes.  You will have to come back to this page and re-enter your username and password in order to refresh your note list.


I am not saving passwords:


At this time I am not saving anybody's password.  I am only saving the list of notes.  Once I update my security model I will provide an option where you can choose to save your password to facilitate automatic updating.  Until then, you will have to manually enter your username and password every time you wish to refresh your list of notes.


Activating the Prosper API for your account:


In order to use the Prosper API you have to have it turned on.  You can do this by logging into Prosper's Website and going to "My Account"->"Settings" Then clicking the "Edit" link on the API box.   You can also try and follow this direct link.  You have the option of making your Prosper API account read only.  If you are not currently bidding on loans with the API then I would recommend you do so as an added measure of security.  (And if you are really paranoid, change your API password immediately after downloading your note listing into Prosper-Stats.)  DO NOTE MAKE YOUR API PASSWORD THE SAME AS THE WEB LOGIN PASSWORD.


Monday, May 27, 2013

Migration to the new Prosper API underway

I started working a major rewrite of prosper-stats.com awhile ago, but I have not had enough free time to really get that project completed.  Since the search data has not been updated for a few weeks (ever since Prosper shut off the legacy API) I decided to take a minor detour and get the Loan Analysis section of the website working again using the new API so that the site would at least be functional until I can find more free time to finish up the larger rewrite.

To that end I created a data conversion tool that does a best fit conversion of the data from the new API into a format similar to the old API.  There are some changes in how data is represented and even in what data is provided in the new API vs the old API so it isn't a perfect fit.  Some of the calculations had to change to use what data was available and some of the search fields won't function correctly any more since I can't recreate parts of the data.  I have also not updated the "active listings" tab of the Loan Analysis tool to use the new API, so that is not returning any results yet either.

Hopefully this update makes prosper-stats.com at least somewhat useful again until I can finish up my rewrite and start introducing new tools that will replace the Previously Late loan tool and add some new functionality.

Thursday, March 14, 2013

Bid Data No Longer Available


Prosper has decided to stop making bid information available.  Currently all of the information I have about lenders is deduced from bids.  Lenders are tied to bids, bids are tied to listings, and listings that get funded are tied to loans.  Without the bid data this association falls apart and it will be impossible to track lender investments going forward.

Prosper has stated that they are doing this for privacy reasons.  I can respect the fact that some lenders feel that to much information was being divulged   However, I have to strongly disagree with the choice Prosper has made to address these lenders' privacy concerns.  There were other approaches that could have been taken that would have struck a better balance between the desire for data transparency of some lenders with the desire for increased privacy of other lenders.  First off, Prosper could simply have removed the linkage between lenders and bids and published anonymous bid data.  Doing this would still have allowed better transparency of listing availability. Lenders could use this anonymous bid data to properly take into account timing constraints on their bidding strategies (because lets be realistic, with the current volume of listings, timing is critical since listings go quickly) as well as allowing retail investors to see if they even have a chance of investing in the current environment by knowing if loans are being eaten up in 1 or 2 bids.  Taking this one step further, Prosper could have created an opt-out mechanism so only lenders who choose to opt-out would have their bid data published anonymously.  This would have allowed for data analysis just like we have today (using tools such as those that my site currently provides) for a sizable portion of the Prosper data set, while still allowing privacy for those lenders who want it.  Both of these options were presented to Prosper months ago, yet with no notice to their user community, they just started removing the availability of bid data.  I find this to be a rather upsetting turn of events and wish that Prosper had handled it differently.


Now that my personal opinion is out of the way, let me clarify how this will impact my site in the short term:

Top Lenders will become more and more out of sync with reality as time goes on since no new loan information will get incorporated into lender portfolios.  I will probably just remove this section of the site after awhile.

The Lender Activity page is still partially working since bids have only been removed from the data export at this time and the open/closed listing information is pulled from the API.  However the API is supposed to be changed shortly as well.  This will effectively render this page useless and I will delete it some time after the API is changed.

Previously Late Loans will continue to let you track loans that I know about, however no new loans will show up and you will have to go back to whatever your previous system was for tracking late loans (good luck with this one!).

Loan Analysis searches that use screen names and note size filters will only return results for older loans where I have bid data.


Now to address how this will impact my site long term:

I don't know what I am going to do!  There is another major change that is supposed to come out shortly.  Prosper is updating their API to have additional data elements for all listings.  This change is not going to be incorporated into the data export.  This means I have two options.

Option 1 is to leave my site mostly as is, which means half of it will have outdated information and the Loan Analysis will only support a subset of the data elements that Prosper will have available (i.e. those available in the data export).

Option 2 is to rewrite portions of my site to pull all the data from the API and allow searching on all available data elements.  As part of this I would abandon the Top Lenders and Lender Activity pages.  Using the new API I should theoretically be able to write new portfolio management tools as well (i.e. tools similar to the Previously Late Loans tool, as well as new ones) by allowing people to register their new API credentials and querying their personal note portfolio through the new API methods.

My availability of free time will be a major deciding factor on what happens going forward.  I have some personal matters going on that are taking up a lot more of my time now then I did when I originally put this site together, so it is a wait and see for me as much as it is for everyone else.

P.S. I have been in touch with Prosper and have access to a beta version of the API and have started to plan out how I will change the site if time permits...

Saturday, February 2, 2013

Folio Exclude Screen Name

When using Folio I typically don't like to bid on notes for loans that I already am invested in.   In the past I would open up the original Prosper listing for everything I was bidding on and see if the yellow banner appeared letting me know if I had already invested on the loan or not.  Lately this banner has not been appearing correctly for me and I accidently picked up a couple duplicates.

Since I am not the only one who doesn't like picking up duplicates I decided to save myself the effort of even needing to look at the original Prosper listings by adding a new Folio specific search filter to remove any loan that I have already invested in.  The new search option is "Folio Exclude Screen Name".   You can put any lender's screen name into this field and any loans that lender help fund will be excluded from the results.  Currently I don't have access to any information about notes bought/sold on Folio, so this won't prevent you from buying the same note on Folio twice, but it can help prevent you from buying duplicates of notes you picked up at origination.

Monday, January 14, 2013

How I calculate returns

High level formula I use for calculating returns


  1. Calculate an estimated loss rate for each of the different "Late" categories based on historical data.
  2. Calculate the gross return of each individual loan.
  3. Calculate the estimated rate of loss for each individual loan.
  4. For all the loans in any particular query I then calculate the dollar weighted and time weighted average of both the gross return and annual loss.
  5. Subtracting the time weighted average loss from the gross return to come up with an overall net return as a yearly APR.


There are a few nuances for each of these steps that people have asked about and so I am going to provide more details for each of these steps below.

Calculation for loss rate of "Late" categories


To come up with an estimated loss rate I first found the percentage of loan value that has historically been lost during a default, this was about 85%.  Then I calculated how often a loan goes to charge off after going late.  I calculated this separately for each late value and rounded off the results for ease of use (since this is an estimate).  The charge off rates are about 60% for late, 85% for 1 month late, 90% for 2 months late, and 95% for 3+ months late.  I multiplied the two together to come up with a loss estimate.  For example, if you had a loan with $10.00 in principal remaining and it was Late, I would estimate a loss of $5.10 ($10.00 * .6 * .85). The same loan at 3 Months late would be an estimated loss of $8.08 ($10.00 * .95 *.85).  Anything that has defaulted gets completely written off.

Calculation for gross return and rate of loss


The annual rate of return and annual rate of loss for any loan is not as straight forward of a calculation as one would think.  First off, Prosper does not provide the actual return for any loan in the data export, what they do provide is a history of the principal balance.  Using this history I go back and calculate an estimate of how much interest was accumulated and payed out between each of the changes in principal balance.  I then take the collected interest and average daily balance to generate a gross return for the loan.  I do not take accrued interest into account in any of my calculations.  The calculation pretends like the most recent month doesn't exist until that month's payment is either made or missed.

The loss rate is calculated in a similar manner but the estimated loss in principal (based on the formula above in "Late" categories) is used against the average daily balance instead of the accumulated interest to come up with an estimated loss rate.

Dollar weighted and time weighted averages


Dollar weighted returns are turned on by default, but the loan analysis tool has a check box to turn them off and I highly recommend you think about what it is you are trying to calculate/analyze to figure out if dollar weighting the return is appropriate for you.  Dollar weighting the returns will make it so that a $10,000 loan will have more of an impact on the total return than a $5,000 loan will.  This might be appropriate if you are looking at an individual investors portfolio and you want to see the return based on how much they actually invested in each loan.  However if you are trying to come up with an investing scheme and you plan to buy $25 notes (or any dollar amount that is not a percentage of the total loan amount), then you might want to analyze the data without dollar weighted averages, since your note sizes will not be weighted by the size of the loan.

Time weighting is what converts the return and loss rate into a yearly rate.  This can have interesting effects for notes that are very young, especially when calculating a loss estimate.  Take the example of a note that never pays and defaults at month 4.  It has a 100% loss over 4 months that when converted to a yearly rate of loss would be 300%.  Obviously it is impossible to lose 300% on any individual note, so taken out of the larger context of calculating a return for a pool of loans, this might look like something is wrong.  However in the context of a larger pool of notes, this behavior is actually correct.  It is helping to estimate future losses that will occur in other notes before the year is out.  So if this note was added in to a portfolio of young HR notes that were all current, it balances out the estimated 30% return those notes have, to give you a more realistic view of what you can expect the returns for your portfolio to be.  As you can see, the accuracy level of the estimated yearly return can be quite suspect for extremely young loans or for a very small subset of loans, however the accuracy level becomes more reliable when the size and age of the loan pool being analyzed increases.

The final net return (which should be looked at along with the loss estimate)


This is the easiest part of the whole calculation.  You just take the yearly gross return and subtract the loss estimate to get the net return.  Nothing magical happens here.  The only reason this step stands out as significant is because the intermediate loss estimate that goes into calculating the net return is a meaningful number that should be taken into account along with the net return when analyzing the data.   Consider a pool of loans returning 8% with a loss estimate of 0% and a pool of loans returning 8% with a loss estimate of 8%.  Which is better?  Both are netting you the same dollar amount each month, but they are getting there differently and you should understand how changes to the APR of new loans at Prosper will impact each of these portfolios differently.  Also if you are not in a tax sheltered account the way losses are handled on your taxes can make the after tax return for each of these two portfolios different.



I hope that all makes sense.  If you have any questions and/or would like clarification on anything I mentioned feel free to ask.

Sunday, January 13, 2013

About this blog

I created and maintain the tools on www.prosper-stats.com.  I wanted someplace where I could document how the different parts of the site work, and so I decided to make this blog.  I thought I would go with the blog approach because it would allow other people the ability to respond and hopefully supplement what I post.

There are multiple generic p2p forums and discussion groups out there and I am not trying to make this blog into another one.  My goal is to create a single consolidated place for documentation on how the tools on  www.prosper-stats.com work and to have a place where I can keep people updated on how they are changing.