Published in the British Medical Journal, the Australian researchers reviewed 13 clinical trials and found that the drug did not improve quality of life or reduce disability.
The group warned that instead it increases the odds of liver problems.
The NHS is going to review its guidelines on the subject. However, experts say that before changing medicine, you should consult your doctor.
Back pain affects an estimated 26 million people each year in the UK.
NICE, which currently establishes NHS best practice, recommends paracetamol for both osteoarthritis and lower back pain.
But the quality of the evidence that led to the decisions has been questioned.
The team from Sydney analysed 13 drug trials that involved over 5,000 patients.
Based on their analysis, they concluded that paracetamol was “ineffective” at controlling back pain.
There was however a small improvement with osteoarthritis in the knee or hip, but the impact was so small it wasn’t deemed “clinically important”.
One of the researchers, Gustavo Machado, argued: “Paracetamol is the most widely used over-the counter medicine for musculoskeletal conditions so it is critical that we review treatment recommendations in light of this new evidence.
“In our research, paracetamol for low back pain and osteoarthritis was also shown to be associated with higher risk of liver toxicity in patients.
“Patients were nearly four times more likely to have abnormal results on liver function tests compared to those taking placebo pills.”