Scribd
Upload
11 views15 pages

Task

practice python task

Uploaded by

meenalgarg13jan
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF or read online on Scribd
11 views15 pages

Task

practice python task

Uploaded by

meenalgarg13jan
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF or read online on Scribd
Import necessary I import pandas as pd import numpy as np import seaborn as sns import [Link] as plt from imblearn.under_sanpling import RandonUnderSanpler from sklearn.nodel_selection import train_test_split from [Link] import KNeighborsClassifier from skopt import BayesSearchCv from [Link] import classification_report import warnings Load the data data = pd.read_csv("[Link]") data 3 4 6362615 6362616 6362617 6362618 6362619 743 743 743 743 743 type PAYMENT PAYMENT TRANSFER CASH_OUT PAYMENT CASH_OUT TRANSFER ‘CASH_OUT TRANSFER CASH_OUT amount 9839.64 186428 181.00 181.00 1668.14 339682.13, 931140928 531140928 5000252 5000252 6362620 rows x 11 columns df = [Link]() Checking missing values df-isnull().sum() nameOrig oldbalanceOrg 1231006815 17013600 1666544295 21249,00 1305486145 181.00 840083671 181.00 2048537720 41554.00 c7e64e4425 —-339682.12 129008245 631140928 1162922333 631140928 1685995037 85000252 1280323807 85000252 newbalanceOrig 16029636 1938472 9.00 9.00 2988586 9.00 9.00 9.00 9.00 9.00 nameDest 1979787155 2044282225 (553264065 38997010 1230701703 776919290 1881841831 1365125890, 2080388512 873221189 step type amount nameOrig. oldbalanceorg nenbalanceOrig nameDest oldbalanceDest nenbalanceDest isFraud isFlaggedFraud dtype: intea The dataset does not contain any missing values. Checking duplicate values df .duplicated() .sum() @ The dataset does not contain any duplicate values. Dataset Description dF. describe() step amount oldbalanceOrg _newbalanceOrig oldbalanceDest_newbalanceDest count 5362620e+06 5.3626202+06 5.362620e+06 6 362620e+06 6 362620e+06 6.362620e+06 mean 2433972e+02 1,798619e+08 8.338831e+05 8551137e+05 1.100702e+06 —_1,224996e+06 std 14233202+02 6.038582e+05 2.888243e+06 2.924049e+06 3.399780e+06 ——3.674129e+06 min 1,000000e+00 0.000000e+00 0.000000e+00 0.000000e+00 0.000000e+00 _0.000000e+00 25% 1.560000e+02 1.338957e+04 0.000000e+00 0.000000e+00 0.000000e+00 -0.000000e+00 50% 2390000e+02 7.487194e+04 1.420800e+04 0.000000 +00 1,327057e+05 -2.146614e+08 75% 3350000e+02 208721Se+0§ 1.073152e+05 1.442584e+05 9.430367e+05 —_1.111909e+06 max 7.430000e+02 9244552e+07 5.958504e+07 4958504e+07 3560159e+08 ——3.561793e+08 # Rename the ‘oldbalancedrg’ colunn to ‘oldbalanceorig' [Link](colunns={'oldbalancedrg': ‘oldbalancedrig'}, inplace=True) Distribution of Transaction types ‘transaction_count_by_type = éf['type’].value_counts() # Plot a pie chart [Link](Figsize-(8, 8)) plt-pie(transaction_count_by type, labels-transaction_count_by [Link], autopct="%1 [Link]( ‘Distribution of Transaction Types") plt-show() Distribution of Transaction Types CASHIN TRANSFER CASH_OUT PAYMENT Count of Fraud / Non Fraud and Flagged Fraud / Non flagged fraud transactions # Display the count of records where isFraud is 1 and @ separately fraud_counts = df['isFraud' ].value_counts() print("Fraudulent Transactions (=1):", fraud_counts[1]) print("Non-Fraudulent Transactions (=@):", fraud_counts[@]) # Display the count of records where isFlaggedFraud is 1 and @ separately flagged_fraud_counts = df['isFlaggedFraud’ }.value_counts() print("\nFlagged Fraud Transactions (=1):", flagged_fraud_counts[1]) print("Non-Flagged Fraud Transactions (=€):", flagged fraud_counts[@]) Fraudulent Transactions (=1): 8213 Non-Fraudulent Transactions (=@): 6354407 Flagged Fraud Transactions (=1): 16 Non-Flagged Fraud Transactions (=@): 6362604 Now to enhance data cleanliness, we'll exclude rows corresponding to customers starting with “M" (Merchants), as there is no available information for them. Deleting the rows # Delete rows where naneDest starts with ‘M’ df_filtered = df[~df['nameDest'].[Link]('M')] Count of Fraud / Non Fraud and Flagged Fraud / Non flagged fraud transactions in updated data # Display the count for isFraud and isFlaggedFraud in the updated data is_fraud_counts = df_filtered[''isFraud’ ].value_counts() is_flagged_fraud_counts = df_filtered[ 'isFlaggedFraud" ].value_counts() print("Fraudulent Transactions in Updated Data:") print(" Fraud (isFraud=1):", is_fraud_counts[1]) print(" Non-Fraud (isFraud=):", is fraud_counts[@]) print("\nFlagged Fraud Transactions in Updated Data:") print(" Flagged Fraud (isFlaggedFraud=1):", is_flagged_fraud_counts[1]) print(" Non-Flagged Fraud (isFlaggedFraud=0):", is flagged_fraud_counts[@]) Fraudulent Transactions in Updated Data: Fraud (isFraud=1): 8213 Non-Fraud (isFraud=0): 4202912 Flagged Fraud Transactions in Updated Data Flagged Fraud (isFlaggedFraud=1): 16 Non-Flagged Fraud (isFlaggedfraud=0): 4211108 After removing rows for customers starting with “M," we find that the counts of fraud cases and flagged fraud ‘cases remain unchanged, indicating no fraud cases among these excluded rows. Update the changes in the dataset # new dataset df = df filtered Distribution of Transaction types in the updated data ‘teansaction_count_by_type = df[‘type’].value_counts() 4 Plot a pie chart [Link](figsize=(8, 8) [Link](transaction_count_by_type, labels=transaction_count_by_type.index, autopct="%? [Link]( "Distribution of Transaction Types’) pit. show() Distribution of Transaction Types TRANSFER CASH_IN This indicates that the transaction types associated with merchants were exclusively payments, as the entire payment category is removed from the pie chart after the removal of merchant data. Check for any rows where both ‘isFraud' and ‘isFlaggedFraud! are 1. same_fraud_flagged_fraud_1 = df{(d#["isFraud'] == 1) & (df {‘isFlaggedFraud' ] if not same_fraud_flagged_fraud_ empty: print ("Count of Ros where isFraud and isFlaggedFraud are both 1:", len(same_frauc els print("No rows found where isFraud and isFlaggedFraud are both 1.") Count of Rows where isFraud and isFlaggedFraud are both 1: 16 For all flagged fraud cases where the value is 1, the corresponding ‘isFraud' value is also 1. Hence, the ‘isFlaggedFraud! column can be removed from the dataset. Remave flagged fraud column [Link]("isFlaggedFraud', axis=1, inplace=True) shape of the updated data [Link] (4211125, 10) Count of fraud cases categorized by transaction type # Define a List of types all_types = df['type" J unique() 4 Tterate through each type and count the rows for transaction_type in all_types: # Filter the rows based on the specified conditions filtered_data = df[(df["type’] == transaction_type) & (Af[isFraud"] == 1)] # Count the number of rows in the filtered dataset count_of_rows = len(filtered_data) print(f"No. of cases where type=(transaction_type), and Fraud=1: {count_of_rows}") No. of cases where type=TRANSFER, and Fraud=1: 4097 No. of cases where type=CASH_OUT, and Fraud=1: 4116 No. of cases where type=DEBIT, and Frau e No. of cases where type=CASH_IN, and Fraud=1: @ As there are no instances of fraud associated with debit and cash-in transactions, we can safely eliminate all data related to these transaction types from the dataset. Remove "DEBIT" and “CASH IN" Transactions df = df[(df['type'] I= 'DEBIT') & (df["type'] I= ‘CASH_IN')] df. shape (270489, 10) df step 2 3 15 19 24 6362615 6362616 6362617 6362618 6362619 743 743 743 743 43 type TRANSFER CASH_OUT CASH_OUT TRANSFER TRANSFER CASH_OUT TRANSFER CASH_OUT TRANSFER CASH_OUT amount 181.00 181.00 22913394 21531030 31168589 339682.13, 931140928 931140928 5000252 5000252 2770409 rows x 10 columns nameOrig oldbalanceOrig newbalanceOrig 1305486145 840083671 c90so8o434 1670993182 1984094095 786484425 1529008245 1162922333, 1685995037, 1280323807 Outliers representation using Box plot # Columns to check for outliers colurns_to_check = ['anount', ‘oldbalanceOrig', ‘newbalanceOrig', ‘oldbalanceDest', # Set up subplots fig, axes plt. subplots (nrow: [Link]( ‘Outlier Detection - Box Plot’, fontsiz 181.00 181.00 15325.00, 705.00 10835.00 339682.13, 931140928 931140928 5000252 5000252 ) # Iterate through columns and create box plots with grid for i, colunn_to_check in enumerate(columns_to_check # Box Plot with grid sns. boxplot (a axes(i].grid(True, linesty! ces[i], x-df[column_to_check]) 2 alpha=@.7) # Add grid axes[i].set_title(f'Box Plot - {colunn_to_check}') # Adjust Layout plt.tight_layout(rect=[0, @.03, 1, 2.95]) # Show plots [Link]( ) oc oc oc oc oc oc oc oc oc oc nameDest 553264065 38997010 (476402209 1100439041 932583850 c77681929¢ 1881841831 1365125890 2080388513 873221189 , ncols=len(colunns_to_check), figsize=(15, 5)) Outer Detection - Box Plot Detect and eliminate outliers as per the presented box plots. 1) In "Amount" column I # Set the threshold for outliers in crores outlier_threshold = 8 4 Identify and print the outlier values in the ‘amount' column outlier_values = df[df[‘amount'] > outlier threshold * 1e7][['type’, ‘amount’, ‘isFray print("Out1ier values in the ‘amount’ column:") print(outlier_values) Outlier values in the ‘amount’ column type amount isFrauc 3686583 TRANSFER 92445516.64 @ f # Remove outliers df = df[df['anount'] <= outlier_threshold * 1¢7] 2) In “oldbalanceOrg" column. n # Set the threshold for outliers in crores outlier_threshold = 5.5 # Identify and print the outlier values in the ‘oldbalancedrg’ coLunn outlier_values = df{df{‘oldbalancedrig"] > outlier_threshold * 1e7][["type’, ‘anount’, print("Outlier values in the ‘oldbalancedrig’ colunn:") print(outlier_ values) Outlier values in the ‘oldbalanceOrig' column: type amount oldbalancedrig isFraud 5996399 TRANSFER 10000000.@ © 59585040.37, 1 6362852 TRANSFER 10000000.0 © 57316255.05 1 Tr 4 Remove outliers df = df[dF['oldbalanceorig’] <= outlier threshold * 1¢7] 3) In “newbalanceOrig” column No outliers appear to be present in the "newbalanceOrig" column based on the provided box plot 4) In “oldbalanceDest" column @ Set the threshold for outliers in crores outLier_threshold = 3.5 4 Identify and print the outlier values in the ‘oldbalanceDest" colum outlier_values = df[df[‘oldbalanceDest’] > outlier_threshold * 1e8][['type", ‘amount’, print("Out1ier values in the ‘amount’ column:") print(outlier_values) Outlier values in the ‘amount’ column ‘type amount oldbalanceDest isFrauc 5380607 CASH_OUT 194946.47 3,551855e+08, e 5486389 TRANSFER 635405.82 3,553805e+08, @ 592018@ CASHOUT 163389.56 3.560159e+08, e # Remove outliers df = df[df['oldbalanceDest'] <= outlier threshold * 1¢8] 5) In “newbalanceDest" column # Set the threshold for outliers in crores outlier threshold = 3.5 # Identify and print the outlier values in the ‘newbalanceDest’ colum outlier_values = df[df['newbalanceDest'] > outlier threshold * 1e8][['type’, ‘anount', print("Outlier values in the ‘amount’ colunn:") print (outlier_values) Outlier values in the ‘amount’ column: type amount newbalanceDest isFraud 4124250 TRANSFER 3325263.90 355553416.3 6 4133583 TRANSFER 30638354.89 355553416.3 e 4 Remove outliers if = df[df[’newbalancedest'] <= outlier_threshold * 1¢8] shape of the dataset after the removal of outliers print("After renoving outliers the updated dataset is : df After removing outliers the updated dataset is :

You might also like