Simplifying Data Imports in Django with Python: The ImportFileOperation Class

Aduda Shem

• 3 min read
Simplifying Data Imports in Django with Python: The ImportFileOperation Class
django

In Django applications, there often comes a time when you need to handle the bulk import of data, such as importing data from an Excel spreadsheet. To make it easier, there's an ImportFileOperation. This class is designed to handle the import of data from an uploaded file and save it to the appropriate Django models.

Importing Required Libraries

import json
import pandas as pd
from django.apps import apps
from django.core.files.uploadedfile import UploadedFile
from django.db import transaction
from distributor.models import Distributor
from distributor.models import DistUsers
from distributor.models.distributor import RetailerProfile
Importing Required Libraries
  • apps: Part of Django, used to get access to models.
  • json: Python's built-in library for working with JSON data.
  • pandas:  powerful library for data manipulation and analysis.
  • UploadedFile: Django class for handling uploaded files.
  • transaction: Django's transaction management for database operations.

Class Initialization

class ImportFileOperation:
    def __init__(self, uploaded_file, user_instance=None, distributor_instance=None, **kwargs) -> None:
        self.uploaded_file = uploaded_file
        self.user_instance = user_instance
        self.distributor_instance = distributor_instance
        self.excel = None

        if user_instance is not None:
            self.user_id = user_instance.id

        if distributor_instance is not None:
            self.distributor_id = distributor_instance.id

        self.check_file()
Class Initialization


class constructor (__init__), the ImportFileOperation class takes several parameters:

  • uploaded_file: (an Excel spreadsheet) to be processed.
  • user_instance: An optional user instance.
  • distributor_instance: An optional distributor instance.
  • **kwargs: Additional keyword arguments.

The constructor initializes the class attributes, including the uploaded file (uploaded_file), user instance (user_instance), distributor instance (distributor_instance), and an attribute called excel to store the parsed Excel data.

File Checking

check_file method verifies whether a file has been uploaded.

    def check_file(self):
        if self.uploaded_file is None:
            return "File upload not found", False
        return None
File Checking

Reading and Parsing Excel Data

read_file method reads and parses the Excel data based on the provided model_name:

    def read_file(self, model_name):
        file_status = self.check_file()
        print("file_status", file_status)
        if isinstance(file_status, str):
            return file_status

        print("self.uploaded_file", self.uploaded_file)
        excel = pd.read_excel(self.uploaded_file)
        print("excel records read file", excel)

        self.excel = excel
        print("EXCEL", excel)
        return self.write_to_model(model_name)
Reading and Parsing Excel Data

It first checks if a file exists using the check_file method.

  • If the file is found, it reads the Excel data into a Pandas DataFrame (excel).
  • The parsed Excel data is stored in the excel attribute for later use.
  • It then calls the write_to_model method to save the data to the appropriate Django model based on the model_name

Saving Data to Django Models

    def write_to_model(self, model_name):
        self.model = apps.get_model(app_label="distributor", model_name=model_name)
        if self.excel is None:
            return "Failed to read the Excel file", False

        if model_name == "PCategory":
            return self.save_category_data()
        elif model_name == "Product":
            return self.save_product_data()
        elif model_name == "SalesMan":
            return self.save_salesman_data()
        elif model_name == "Retailer":
            return self.save_retailer_data()
        elif model_name == "Brand":
            return self.save_brand_data()

        return "Model for saving not implemented", False
Saving Data to Django Models

write_to_model

  • uses apps.get_model to dynamically fetch the Django model based on the provided model_name.
  • If the excel attribute is None , it returns an error message.
  • Depending on the model_name, it calls specific methods (e.g., save_category_data) to handle the data-saving logic for that model.

Saving Category Data

    def save_category_data(self):
        try:
            with transaction.atomic():
                for index, row in self.excel.iterrows():
                    category, created = self.model.objects.get_or_create(
                        distributor_id=self.distributor_id,
                        name=row['CATEGORY'],
                        defaults={
                            'brief_description': row['DESCRIPTION'],
                        }
                    )
        except Exception as e:
            return f"Error saving category: {str(e)}", False

        return "Category saved successfully", True
Saving Category Data

The method uses a transaction to ensure data consistency. It iterates through the parsed Excel data and uses get_or_create to either retrieve an existing PCategory object or create a new one based on the provided attributes.

[A transaction is a database operation unit that ensures data integrity, consistency, and reliability. It follows ACID properties, guaranteeing that all its actions are atomic, consistent, isolated from other transactions, and durable after completion.]