DISQUS

JP · 3 months ago

Thanks for sharing, but where do you enter this line : git clone git://github.com/baz/app-sales-machine.git
???

JP · 3 months ago

sorry for this ub uestion, I've jsut found Git is a version control system.
I've successfully installed your app. Is there a way to test the application without waiting for one day to test if one settings are good ?

JP · 3 months ago

I tried to upload a weekly report and I got this error :

Traceback (most recent call last):
File "/base/python_lib/versions/1/google/appengine/ext/webapp/__init__.py", line 509, in __call__
handler.post(*groups)
File "/base/data/home/apps/appstoresalesmachine/1.336283913765627149/handlers/admin.py", line 32, in post
tarball = tarfile.open(fileobj=file_data)
File "/base/python_dist/lib/python2.5/tarfile.py", line 1153, in open
raise ReadError("file could not be opened successfully")
ReadError: file could not be opened successfully

Ben · 3 months ago

Thank you for sharing. I was able to deploy ASM and test the jobs successfully, however, I am experiencing some difficulty uploading a tarbell of my daily reports.

1. I downloaded my last 4 daily reports from iTunes Connect into a new directory.
2. I changed to the new directory and executed: "tar cvf reports.tar *.txt"
3. I opened my app's home URL and attempted to upload reports.tar which resulted in:

Traceback (most recent call last):
File "/base/python_lib/versions/1/google/appengine/ext/webapp/__init__.py", line 509, in __call__
handler.post(*groups)
File "/base/data/home/apps/headcasegamesappsales/2.336355808577417080/handlers/admin.py", line 37, in post
report_persister.persist(tarinfo.name, file_buffer)
File "/base/data/home/apps/headcasegamesappsales/2.336355808577417080/processors/report_persister.py", line 15, in persist
parsed_data = report_munger.AppStoreSalesDataMunger().munge(content, settings.SETTINGS['base_currency'])
File "/base/data/home/apps/headcasegamesappsales/2.336355808577417080/processors/report_munger.py", line 98, in munge
date = parsedSales[-1]['date']
IndexError: list index out of range

Using the GAE Dashboard Datastore Data Viewer it appears that the reports were not added to RawReport.

Can you advise?

Thank you,
Ben

Basil Shkara · 3 months ago

Sounds like you're doing everything right, but that error you are seeing indicates that the data in your reports could not be parsed.

Not sure what to tell you except to ensure that the tar file is a flat structure and contains only text files as you received them from iTunes Connect. You can debug it by placing some log statements in processors/report_munger.py which the stack trace describes.

JP · 3 months ago

Sorry to spal, Iwas just uploading the .gz downloaded from itunes conenct, and it seems you have to unzip and then tar the reports.

Basil Shkara · 3 months ago

Yes create your own tar archive and upload that. It's rather primitive at the moment about what it expects.

To test your settings, you can visit the following URL:

<your application name>.appspot.com/jobs/pull_report

That will download yesterday's iTunes Connect report, parse it and store it in the data store, provided yesterday's report currently exists. If it doesn't you will get a stack trace. :(

If no stack trace appears, it probably worked. Log into your GAE account and examine the data store to see if it downloaded correctly.

Like I said, rather primitive at the moment but eh it works.

Dave · 3 months ago

Great thanks for this really nice, the only issue I have is I dont seem to be getting the email through. I have altered the details, the CRON job runs with no errors in the error log, but I get no email. I check the quota details and I see no mail api calls either so not sure what is up here?

Basil Shkara · 3 months ago

Make sure that reports already exist in the data store. If none exist then the email_report job will silently fail.

You can test a local instance by filling out settings.py and then running it locally using the Google App Engine Launcher.

Pull the latest report:

http://localhost:8080/jobs/pull_report

Or import a tarball with any existing reports from the admin page.

Once there is content inside the data store hit this URL:

http://localhost:8080/jobs/email_report

And watch the GAE Log Console for output regarding email.

It should dump out the attributes of your email (from, to, subject) and tell you that 'You are not currently sending out real email.'

Dave · 3 months ago

Yea info seems to be uploading ok as far as i can see. It just doesnt then try and upload.. wondering if it is something to do with the email addresses entered, I dont see any error logs that point to any errors

JP · 3 months ago

Thanks for your replies.
So here is some feedback. I received the first automated email and I can only say it's awesome.

A small problem about the graph : to initialize my data, I uploaded my weekly reports. It worked fine. Now the data are automatically updated daily. Let's say I sell 100 apps a day. On the left part of the graph where data have been imported from weekly reports, the graph is at 700 sales. And now that the sales are daily, the graph goes down to 100. So it looks like the sales have decreased a lot, but they have not. Do you see what I mean ?

Basil Shkara · 3 months ago

I don't parse weekly reports at the moment.

You would be better off uploading daily reports for accurate reporting and graph generation.

If you upload weekly reports, the figures and graphs are likely to be wrong.

JP · 3 months ago

Does the database can say if a figure is imported from a weekly or daily report ? In that case that would be easy to make an acceptable graph by using the (number sales / 7) for weekly figures in the graphs.

Basil Shkara · 3 months ago

No the data store doesn't record whether the report came from a weekly or a daily. The parser does not understand weekly reports and tries to ignore them. You could try adding support for this if you want. :) Start in processors/report_persister.py.

JP · 3 months ago

OK thanks :)

Basil Shkara · 3 months ago

I should add that weekly reports only report the total number of sales/upgrades for a 1 week period. You do not get a daily breakdown which is necessary for the graph generation. If weekly reports were captured, it would mean a separate graph would have to be generated to represent those reports. Just FYI for you or others looking for weekly report support.

uprise78 · 3 months ago

First off, thanks for the sweet machine. It works great for one app but for some reason my other app doesn't send off an email. I checked out the datastore and there are upgrades for both SKUs but there are no sales for one of the SKUs. Any idea where I should start debugging the issue? I am guessing whereever the rawdata gets parses is probably the culprit.

Basil Shkara · 3 months ago

I just tested this and it does correctly parse and store sales data for multiple SKUs.

However it will not send any email to a product which does not have any existing sales data in the data store.

If that is your issue, then I just committed a patch that fixes this issue here: http://github.com/baz/app-sales-machine/commit/...

If the issue is that it is not correctly parsing and storing sales data from your reports then I have not been able to replicate this. If you would like to try debugging it yourself, the data parsing occurs in processors/report_munger.py. You may want to start from handlers/admin.py though.

James · 2 months ago

Hi, thanks for sharing! I've been working on getting this running for the past day or so, it's been a crash course in Python and Google App Engine (GAE is very exciting, haven't used it before). Ran into all sorts of problems getting everything sorted out on my dev machine and then uploaded to google. Among the issues I've run into are one where /jobs/pull_report doesn't work during certain times of day (has something to do with time zones, tries to pull the report for today instead of yesterday after midnight GMT), another where my app refused to work on google's server (i accidentally created the app w/ authentication, big no-no), and a third where /jobs/pull_report would give an error about _ctypes not being defined (only in dev, not included in my version of python, but exists on server). Anyway, I've overcome almost all of them now, but I'm just short of being able to get the app to send the report email.

I've uploaded a tarball of my reports, and hit /jobs/email_report, but get this error:

local variable 'sales_start' referenced before assignment
Traceback (most recent call last):
File "/base/python_lib/versions/1/google/appengine/ext/webapp/__init__.py", line 507, in __call__
handler.get(*groups)
File "/base/data/home/apps/jjappsalesmachine/1.336674502001868990/jobs/email_report.py", line 110, in get
'sales_start': self._date_string(sales_start),
UnboundLocalError: local variable 'sales_start' referenced before assignment

Been trying to figure out what's going on here. I pulled down your initial commit of this project and tried that, everything appeared to work (no stack trace), but no email received, in dev or on the server. The above error is with the latest version of the code, and occurs in both my dev machine and in production. I've tinkered and toyed and reverted and started over enough times now, and while the learning is fun, I figure it might be time to see if anybody else has some insight.

Any hints would be greatly appreciated!

James

Basil Shkara · 2 months ago

That stack trace indicates that there is no sales data in your data store.

Log into your GAE dashboard and open the Data Viewer.
Is there any data in the Sale table?

Also you may want to check if your app's SKU is accurate. (Does the value in settings.py match the SKUs shown in Apple's reports?)

James · 2 months ago

I checked, and there is data in the data store.

Initially, my settings.py was configured with 3 SKUs, but my sales report for yesterday only had entries for 1 SKU, so I modified the settings to only reference the 1 SKU.

I now get this error locally when I clear the data store, run pull_report, then email_report:

Traceback (most recent call last):
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/webapp/__init__.py", line 507, in __call__
handler.get(*groups)
File "/Users/james/Code/Tools/app-sales-machine/jobs/email_report.py", line 105, in get
overall_chart_url, concentrated_chart_url = self.units_chart(pid)
File "/Users/james/Code/Tools/app-sales-machine/jobs/email_report.py", line 202, in units_chart
overall_chart.left.max = max(upgrades) if max(upgrades) > max(sales) else max(sales)
UnboundLocalError: local variable 'upgrades' referenced before assignment

Thanks for the help!

Basil Shkara · 2 months ago

I've pushed some changes which should fix your issues.

Update your working copy and try it again. Let me know how you go.

James · 2 months ago

ok, very cool. i pulled down the latest and tried it. i got an error about an argument to max() being an empty sequence, but i fixed this in email_report.py, i assume since my apps do not have any upgrades yet

email_report.py (starting on line 205)

if upgrades:
overall_chart.left.max = max(upgrades) if max(upgrades) > max(sales) else max(sales)
else:
overall_chart.left.max = max(sales)

James · 2 months ago

Oh, and besides the error I fixed, everything is working fine now! You are a scholar and a gentleman, thanks!

James · 2 months ago

Hi again,

So I got the emails to send, but I noticed that for 2 of my 3 SKUs, there was no data (in the email or even in the datastore). I think I fixed the problem:

report_persister.py, starting on line 45:

# Store sale and upgrade data in separate tables
for product in sales:
sale_store = models.data.Sale()
_store_data(product, sale_store, date)
sale_store.put()

for product in upgrades:
upgrade_store = models.data.Upgrade()
_store_data(product, upgrade_store, date)
upgrade_store.put()

def _store_data(product, store, date):
store.income_revenue = float(product['incomeRevenue'])
store.income_units = product['incomeUnits']
store.revenue_by_currency = product['revenueByCurrency']
store.units_by_country = product['unitsByCountry']
store.refund_loss = float(product['refundLoss'])
store.refund_units = float(product['refundUnits'])
store.pid = product['pid']
store.report_date = date

I hope this is helpful!