Google drive downloading large files with C# 11

 in .net Core / C# / Google Drive API  tagged .net core / C# .net / Google API by
Google Drive API

Are you working on an application where you would like to download large files from Google drive? This simple tutorial will show you how to download large files by chunks.

Setup

The first thing you will need to do is create a new project on Google developer console. For this project i will be using a console application so we will create a native client. Note this code will NOT work for a web browser application.

You will also need the NuGet package Install-Package Google.Apis.Drive.v3 you will also need to grab my sample class for authencating to Google drive Oauth2Authentication.cs

Code

class Program
    {
        private const string _pathToCreds = @"C:\creds\client_secret.json";


        async static Task Main(string[] args)
        {
            var service = auth.Oauth2Example.GetDriveService(_pathToCreds, "test",
                new[] {Google.Apis.Drive.v3.DriveService.Scope.Drive});

            var request = service.Files.List();
            request.Q = "name='reallybigfile.zip'";
            var result = await request.ExecuteAsync();

            foreach (var file in result.Files)
            {
                await DownloadFile(file.Id, service);
            }

            Console.WriteLine("Hello World!");
        }


        public static async Task DownloadFile(string fileId, DriveService service)
        {
            const int KB = 0x400;
            var chunkSize = 256 * KB; // 256KB;

            var fileRequest = service.Files.Get(fileId);
            fileRequest.Fields = "*";
            var fileResponse = fileRequest.Execute();

            var exportRequest = service.Files.Export(fileResponse.Id, fileResponse.MimeType);
            var client = exportRequest.Service.HttpClient;

            //you would need to know the file size
            var size = fileResponse.Size;

            await using var file = new FileStream(fileResponse.Name, FileMode.OpenOrCreate, FileAccess.ReadWrite);
            file.SetLength((long) size);

            var chunks = (size / chunkSize) + 1;
            for (long index = 0; index < chunks; index++)
            {
                var request = exportRequest.CreateRequest();

                var from = index * chunkSize;
                var to = @from + chunkSize - 1;

                request.Headers.Range = new RangeHeaderValue(@from, to);

                var response = await client.SendAsync(request);

                if (response.StatusCode != HttpStatusCode.PartialContent && !response.IsSuccessStatusCode) continue;

                await using var stream = await response.Content.ReadAsStreamAsync();
                file.Seek(@from, SeekOrigin.Begin);
                await stream.CopyToAsync(file);
            }
        }
    }
}

Conclusion

Simple download gives us a method for downloading large files from google drive.

About Linda Lawton

My name is Linda Lawton I have more than 20 years experience working as an application developer and a database expert. I have also been working with Google APIs since 2012 and I have been contributing to the Google .Net client library since 2013. In 2013 I became a a Google Developer Experts for Google Analytics.

Leave a comment

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

11 thoughts on “Google drive downloading large files with C#”

  • Reply ↓
    Yura

    public static async Task DownloadFile(string fileId, DriveService service, string _downloadFile)
    {
    const int KB = 0x400;
    var chunkSize = 256 * KB; // 256KB;

    var fileRequest = service.Files.Get(fileId);
    fileRequest.Fields = “*”;
    var fileResponse = fileRequest.Execute();

    var exportRequest = service.Files.Export(fileResponse.Id, fileResponse.MimeType); // “application/x-zip-compressed” fileResponse.MimeType
    var client = exportRequest.Service.HttpClient;

    //you would need to know the file size
    var size = fileResponse.Size;

    var file_downloadFile = new FileStream(_downloadFile, FileMode.OpenOrCreate, FileAccess.ReadWrite);
    file_downloadFile.SetLength((long)size);

    var chunks = (size / chunkSize) + 1;

    for (long index = 0; index < chunks; index++)
    {
    var request = exportRequest.CreateRequest();
    var from = index * chunkSize;
    var to = @from + chunkSize – 1;

    request.Headers.Range = new RangeHeaderValue(@from, to);
    var response1 = await client.SendAsync(request);

    if (response1.StatusCode != HttpStatusCode.PartialContent && !response1.IsSuccessStatusCode) continue;

    var stream1 = await response1.Content.ReadAsStreamAsync();
    file_downloadFile.Seek(@from, SeekOrigin.Begin);
    await stream1.CopyToAsync(file_downloadFile);
    }

    }

    Did so, does not work correctly. The file "Test.zip" is downloaded from Google drive, but does not open. Error "Inconsistent file format" when I try to open it.
    What could be the problem?
    Thanks!

    • Reply ↓
      ROBERT CRUZ

      I was having the same problem. It appears that this code solution is utilizing the “export” function from the Google Drive v3 API. The “export” function is utilized only for GSuite files (ex., *.gsheet), in order to convert them to another format, such as PDF. I too was receiving a 403 forbidden error against regular files, e.g., a native CSV file.

      I updated the code a little bit, but the code snippet also works with the “service.Files.Get(fileId)” functionality

      Code sample:

      const int KB = 0x400;
      var chunkSize = 10240 * KB; // 10240KB (10MB);

      var fileRequest = _service.Files.Get(fileResource.Id);
      fileRequest.Fields = “*”;
      fileRequest.SupportsAllDrives = true;
      var fileResponse = fileRequest.Execute();
      var client = fileRequest.Service.HttpClient;

      //you would need to know the file size
      var size = fileResponse.Size;

      await using var file = new FileStream(fileResponse.Name, FileMode.OpenOrCreate, FileAccess.ReadWrite);

      file.SetLength((long)size);

      var chunks = (size / chunkSize) + 1;
      for (long index = 0; index < chunks; index++)
      {
      var from = index * chunkSize;
      var to = @from + chunkSize – 1;

      var request = fileRequest.CreateRequest();
      request.Headers.Range = new RangeHeaderValue(@from, to);
      var response = await client.SendAsync(request);

      if (response.StatusCode != HttpStatusCode.PartialContent && !response.IsSuccessStatusCode)
      {
      //ERROR OCURRED
      Console.WriteLine(response.ToString());
      continue;
      }

      Console.WriteLine($"ChunkIndex: {index}; File Size: {size}; Bytes Downloaded: {from} ({Convert.ToDecimal(from)/(1024.0m * 1024.0m)}) MB; ");
      await using var stream = await response.Content.ReadAsStreamAsync();
      file.Seek(@from, SeekOrigin.Begin);
      await stream.CopyToAsync(file);

    • Reply ↓
      ROBERT CRUZ

      oops – re-tested my code, and there was a bug (it was downloading the getfile metadata and saving that over and over). Here’s the correct code sample:

      const int KB = 0x400;
      var chunkSize = 10240 * KB; // 10240KB (10MB);

      var fileRequest = _service.Files.Get(fileResource.Id);
      fileRequest.Fields = “*”;
      fileRequest.SupportsAllDrives = true;

      var fileResponse = fileRequest.Execute();
      var client = fileRequest.Service.HttpClient;

      //you would need to know the file size
      var size = fileResponse.Size;

      await using var file = new FileStream(fileResponse.Name, FileMode.OpenOrCreate, FileAccess.ReadWrite);

      file.SetLength((long)size);

      var chunks = (size / chunkSize) + 1;
      for (long index = 0; index < chunks; index++)
      {
      var from = index * chunkSize;
      var to = @from + chunkSize – 1;
      var range = new RangeHeaderValue(@from, to);

      //VALIDATE SUCCESSFUL REQUEST
      var request = fileRequest.CreateRequest();
      request.Headers.Range = range;
      var response = await client.SendAsync(request);
      if (response.StatusCode != HttpStatusCode.PartialContent && !response.IsSuccessStatusCode)
      {
      //ERROR OCURRED
      Console.WriteLine(response.ToString());
      continue;
      }

      //IF NO ERRORS, PERFORM DOWNLOAD RANGE
      Console.WriteLine($"ChunkIndex: {index}; File Size: {size}; Bytes Downloaded: {from} ({Convert.ToDecimal(from)/(1024.0m * 1024.0m)}) MB; ");
      file.Seek(@from, SeekOrigin.Begin);
      await fileRequest.DownloadRangeAsync(file, range);

  • Reply ↓
    ROBERT CRUZ

    Hi Linda,

    I’m trying to implement this solution, but I am also getting a 403 (Forbidden) error.

    It’s pretty ambiguous, I’m trying to get additional data to tell me the reason:

    StatusCode: 403, ReasonPhrase: ‘Forbidden’, Version: 1.1, Content: System.Net.Http.DecompressionHandler+GZipDecompressedContent, Headers:
    {
    Vary: Origin
    Vary: X-Origin
    Date: Fri, 21 Aug 2020 18:21:12 GMT
    Cache-Control: max-age=0, private
    X-Content-Type-Options: nosniff
    X-Frame-Options: SAMEORIGIN
    Content-Security-Policy: frame-ancestors ‘self’
    X-XSS-Protection: 1; mode=block
    Server: GSE
    Alt-Svc: h3-29=”:443″; ma=2592000,h3-27=”:443″; ma=2592000,h3-T050=”:443″; ma=2592000,h3-Q050=”:443″; ma=2592000,h3-Q046=”:443″; ma=2592000,h3-Q043=”:443″; ma=2592000,quic=”:443″; ma=2592000; v=”46,43″
    Transfer-Encoding: chunked
    Content-Type: application/json; charset=UTF-8
    Expires: Fri, 21 Aug 2020 18:21:12 GMT
    }

  • Reply ↓
    Rajib Majumdar

    Hi Linda,
    First of all thanks for writing such a nice article.I am a naive programmer, mostly building Forms Application using .NET Framework & ‘C#’, I am getting a compiler error while using the above code, saying:

    CS1487 ‘FileStream’: type used in an asynchronous using statement must be implicitly convertible to ‘System.IAsyncDisposable’ or implement a suitable, ‘DisposeAsync’ method.
    Did you mean ‘using’ rather than ‘await using’?

    I am using .NET Framework 4.5.2, Creating Windows Forms Application.
    After a little research, I came to Know, ‘IAsyncDisposable’ is incorporated in C# 8.0, so I Changed my program language version to the same. Even then, the compiler error refused to go away. then I tried to Implement ‘IAsyncDisposable’ using reference from, ‘https://docs.microsoft.com/en-us/dotnet/standard/garbage-collection/implementing-disposeasync’. Even that failed, can you tell me where I am doing wrong?.
    Thank You.
    [Note: I have tried even .NET 4.8 & C# 8.0 but no luck]