Google Analytics API Pagination (Pages) with Perl Code

Firstly, an important differentiation between Google Analytics and the Google Analytics API:
- Google Analytics can return upto 500 results per page.
- Google Analytics API can return up to 10,000 results per page.
Please note this and check that you actually require pagination.
Secondly, why do you need more than 10,000 results? Think about this point: Do you really need it? Or do you want it simply because you can?
With that said; it is quite simple to paginate through the results within Google Analytics. The key parameters are:
- max-limit used to specify how many results to return in total for each page.
- start-index is used to specify where to start in the "entire" results list. This is kind of like the page numbers at the bottom of Google's natural search - see image above.
Here is the code in Perl (note that you need the GAhandler, based on Patrick Hartman's code, included for download):
[Code]
#!/usr/bin/perl
#Author: Lynchpin
#A script showing pagination with the Google Analytics API. This is required if returning results greater than 10,000.
use strict;
#NOTE: Get this from the zip file!
use GAhandler;
use XML::Simple;
#SETTINGS
#authenticate with the API to receive token
my $gaUsername = '[GA USERNAME]'; #Username - note that read-level permission is required for the profile, see:
my $gaPassword = '[GA PASSWORD]'; #http://code.google.com/apis/analytics/docs/gdata/gdataDeveloperGuide.htm...
#Recommend the query explorer to find and test the following: http://code.google.com/apis/analytics/docs/gdata/gdataExplorer.html
my $profileId = '[PROFILE ID]'; #Profile ID - see: http://www.lynchpin.com/blog/new-google-analytics-profile-id-location-0
my $gaDimensions = "[DIMENSIONS]"; #Data Dimensions
my $gaMetrics = "[METRICS]"; #Data Metrics
my $gaFilters = "[FILTERS]"; #Data filters - optional
my $gaStartDate = "[START DATE]"; #Start date in the format yyyy-mm-dd
my $gaEndDate = "[END DATE]"; #End date in the format yyyy-mm-dd
my $maxResultsPerPage = 10000; #Number of results per page - max is 10,000 in GA API
print "Begin\n";
my $gahandler = new GAhandler();
print " Get authentication token..\n";
my $token = $gahandler->gaGetToken($gaUsername, $gaPassword);
#Note that the start index should begin at 1, not 0
my $startIndex = 1;
#Declare reused variables
my ($requestUrl, $pagedData, $xmlRaw, $xmlTree, $totalResults);
#Loop through until the start index is more than the total results available
print " Loop through pages..\n";
do {
$requestUrl = "https://www.google.com/analytics/feeds/data?".
"ids=ga%3A".$profileId.
"&dimensions=".$gaDimensions.
"&metrics=".$gaMetrics.
"&start-date=".$gaStartDate.
"&end-date=".$gaEndDate.
"&start-index=".$startIndex.
"&max-results=".$maxResultsPerPage.
"&filters=".$gaFilters;
$pagedData = $gahandler->gaDataFeed($requestUrl, $token);
#GA API puts colons in the element tags... Can't get XML simple to accept a coloned tag
#so replace it with a tag without a colon instead!
$pagedData =~ s/openSearch:totalResults/openSearchTotalResults/g;
$xmlRaw = new XML::Simple(KeyAttr=>[]);
$xmlTree = $xmlRaw->XMLin($pagedData);
#Get the results returned
$totalResults = $xmlTree->{openSearchTotalResults};
#DO SOMETHING WITH PAGED DATA
#...
#/DO SOMETHING WITH PAGED DATA
print " Retrieved ".$startIndex." to ".($startIndex + $maxResultsPerPage - 1)."(max) of ".$totalResults."\n";
$startIndex += $maxResultsPerPage;
} while ($startIndex <= $totalResults);
print "Finished\n";
[/Code]
Comments
iKKnKxpmNIPQ
Just cause it's simple doesn't mean it's not super helpful.
fyXYYHIYmEPRzafdVta
If my preoblm was a Death Star, this article is a photon torpedo.
yewPLPhdrdewpvTrrNN
I canont tell a lie, that really helped.
Add new comment