Question
Given a list of lists containing text values, create a pandas DataFrame and convert the second and third columns to floating-point numbers.
import pandas as pd
table = [
["a", "1.2", "4.2"],
["b", "70", "0.03"],
["x", "5", "0"],
]
df = pd.DataFrame(table)
How can columns 2 and 3 be converted to float values? Can column types be specified while creating the DataFrame, or should the DataFrame be created first and then have its column dtypes changed? How can this be handled dynamically when there may be hundreds of columns, but every individual column contains values of one consistent type?
Short Answer
You will learn how pandas chooses DataFrame column types, how to explicitly convert columns with astype(), and how to safely detect and convert numeric-looking text with pd.to_numeric() when the schema is not known in advance.
Concept
A pandas DataFrame stores data in columns, and each column has a dtype (data type). Common dtypes include:
objectorstringfor textint64for whole numbersfloat64for decimal numbersboolforTrueandFalsedatetime64[ns]for dates and times
In the example, every value except those in the first column is written with quotes. That means Python creates them as strings:
"1.2" # string
1.2 # float
Because the values are strings, pandas initially creates all three columns with an object dtype (or a string-like representation depending on pandas configuration). A value that looks numeric is still text until it is parsed or converted.
For columns that are known to be numeric, convert them explicitly. This prevents later calculations from behaving incorrectly. For example, adding two strings joins them instead of performing arithmetic:
"1.2" +
Mental Model
Think of each DataFrame column as a box with a label describing what it can hold.
- A text box stores characters, even when they look like numbers.
- A number box stores values that can be added, averaged, and compared numerically.
astype(float) tells pandas: “Move every item in this column into a floating-point number box.” If an item cannot be understood as a number, the move fails.
pd.to_numeric() is like a number-reading tool. It reads text such as "70" and "0.03" and turns it into numeric values. It also gives you options for deciding what should happen when it encounters text that is not numeric.
Syntax and Examples
Use astype() when you know the target dtype and all values are valid for that dtype.
import pandas as pd
table = [
["a", "1.2", "4.2"],
["b", "70", "0.03"],
["x", "5", "0"],
]
df = pd.DataFrame(table, columns=["code", "amount", "rate"])
df[["amount", "rate"]] = df[["amount", "rate"]].astype(float)
print(df)
print(df.dtypes)
Output:
code amount rate
0 a 1.2 4.20
1 b 70.0 0.03
2 x 5.0 0.00
code object
amount float64
rate float64
dtype: object
Columns can also be selected by position. pandas uses zero-based positions, so the second and third columns have positions 1 and 2:
df.iloc[:, [1, 2]] = df.iloc[:, [1, 2]].astype()
Step by Step Execution
Consider this small example:
import pandas as pd
df = pd.DataFrame({
"product": ["pen", "book"],
"price": ["1.50", "12.00"]
})
print(df["price"].dtype)
df["price"] = df["price"].astype(float)
print(df["price"].dtype)
print(df["price"].mean())
Step by step:
pd.DataFrame(...)creates apricecolumn from"1.50"and"12.00".- Quotes make both values strings, so
df["price"].dtypeis initiallyobject. astype(float)parses each string as a floating-point value.- The converted values are assigned back to
df["price"]. Assignment is important because conversion methods return a new Series or DataFrame. - The dtype is now
float64. .mean()can now calculate , producing .
Real World Use Cases
Column conversion is common whenever data arrives as text.
- CSV imports: Prices, quantities, and percentages often arrive as strings.
- API responses: JSON fields may contain numeric values encoded as strings, such as
"total": "49.99". - Spreadsheets: User-entered values may need conversion before calculating totals or averages.
- Data cleaning: A column may contain values such as
"12","15", and"not available"; numeric parsing can turn the invalid value into missing data for later review. - Database loading: Convert data to expected dtypes before inserting records into typed database columns.
- Reporting: Numeric dtypes are required for aggregations such as
sum(),mean(), andgroupby()calculations.
Real Codebase Usage
In real projects, developers usually define an expected schema when it is known rather than relying entirely on automatic inference.
Convert known columns together
numeric_columns = ["amount", "rate", "discount"]
df[numeric_columns] = df[numeric_columns].apply(pd.to_numeric, errors="raise")
errors="raise" is useful in data pipelines where invalid input should stop processing immediately.
Convert unknown columns cautiously
When columns may or may not be numeric, attempt conversion one column at a time:
for column in df.columns:
try:
df[column] = pd.to_numeric(df[column], errors="raise")
except (ValueError, TypeError):
pass
This converts columns that are entirely numeric-looking and leaves text columns unchanged. It fits the condition that each column contains one consistent type.
Keep invalid values as missing data
For files that can contain blanks or bad values, preserve the column as numeric and mark problematic entries as missing:
df["amount"] = pd.to_numeric(df["amount"], errors="coerce")
invalid_rows = df[df["amount"].isna()]
This pattern supports validation: process valid values while separately logging or fixing invalid rows.
Common Mistakes
Forgetting to assign the converted result
Most conversion operations do not modify the original column automatically.
# Incorrect: df is unchanged
df["amount"].astype(float)
# Correct
df["amount"] = df["amount"].astype(float)
Converting the text identifier column
The first column contains codes such as "a", "b", and "x". Trying to convert it to float fails.
# Raises ValueError
df = df.astype(float)
Convert only appropriate columns instead.
df[["amount", "rate"]] = df[["amount", "rate"]].astype(float)
Confusing column labels and positions
Without specified column names, pandas labels the columns 0, , and . The second column is label , not .
Comparisons
| Tool or approach | Best use | Invalid values | Result |
|---|---|---|---|
astype(float) | Every value is known to be valid | Raises an error | Exactly the requested dtype |
pd.to_numeric(..., errors="raise") | Strict validation of numeric text | Raises an error | Numeric dtype when possible |
pd.to_numeric(..., errors="coerce") | Cleaning imperfect data | Becomes NaN | Numeric dtype with missing values |
pd.to_numeric(..., errors="ignore") | Rarely recommended | Leaves unconvertible input unchanged | May remain text |
Cheat Sheet
# Convert named columns to floats
df[["amount", "rate"]] = df[["amount", "rate"]].astype(float)
# Convert columns by zero-based position
df.iloc[:, [1, 2]] = df.iloc[:, [1, 2]].astype(float)
# Strict numeric parsing
df["amount"] = pd.to_numeric(df["amount"], errors="raise")
# Bad values become NaN
df["amount"] = pd.to_numeric(df["amount"], errors="coerce")
# Try to convert every fully numeric column
for col in df.columns:
try:
df[col] = pd.to_numeric(df[col], errors="raise")
except (ValueError, TypeError):
pass
# Inspect dtypes
print(df.dtypes)
Key rules:
- Quoted numbers are strings, not numeric values.
- pandas column positions start at
0. - Assign conversion results back to the DataFrame.
- Use
astype()for trusted data andto_numeric()for parsing and validation. - columns can represent missing numeric values as .
FAQ
How do I convert multiple pandas columns to float?
Select the columns and assign the converted result back:
df[["amount", "rate"]] = df[["amount", "rate"]].astype(float)
Why are numeric values stored as object in my DataFrame?
They were likely supplied as quoted strings, such as "70" or "0.03". pandas preserves them as text until you convert them.
Can I set a different dtype for each column in pd.DataFrame()?
Not conveniently with the dtype= argument when building directly from a list of lists. dtype= is appropriate when the DataFrame should use one common dtype. Create the DataFrame, then convert selected columns, or provide actual Python numeric values in the input.
Should I use astype(float) or pd.to_numeric()?
Use astype(float) when all values are guaranteed to be valid floats. Use pd.to_numeric() when parsing text data and you need control over invalid values.
How do I find columns that can be converted to numbers automatically?
Mini Project
Description
Build a small order-data cleaning function. The input resembles data from a CSV export: product identifiers are text, while quantity and unit price are numeric values stored as strings. The function should convert valid numeric columns and calculate a line total.
Goal
Create a cleaned DataFrame with numeric quantity, unit_price, and line_total columns while preserving product codes as text.
Requirements
Create a DataFrame from the provided order rows.
Convert the quantity and unit_price columns to numeric values.
Treat invalid numeric values as missing rather than crashing.
Add a line_total column calculated from quantity times unit price.
Print the cleaned DataFrame and its dtypes.
Keep learning
Related questions
@staticmethod vs @classmethod in Python Explained
Learn the difference between @staticmethod and @classmethod in Python with clear examples, use cases, mistakes, and a mini project.
Add Rows to a Pandas DataFrame in Python
Learn how to add rows to a Pandas DataFrame, why repeated row appends are slow, and when to use loc, concat, or record lists.
Call a Function by Name in a Python Module
Learn how to call a function by name in a Python module using strings, getattr, and safe patterns for dynamic function dispatch.