Saints were beaten 3-0 by Newcastle United on Saturday and lie just one point above the dreaded drop zone.
Assistant manager Carlos Compagnucci and assistant first-team coach Xavier Tamarit have also parted company with the south coast club.
Pellegrino, a former Argentina international, was appointed in June as the successor to Claude Puel.
Southampton have been in the Premier League for six years and are yet to finish lower than 14th. With eight games remaining they have just 28 points.
The Saints, who face Wigan Athletic in the quarter-final of the FA Cup this weekend, have won just one of their last 17 Premier League fixtures.
A statement from the club read: ‘Southampton Football Club can today confirm that it has parted company with First Team Manager Mauricio Pellegrino.
‘Assistant Manager Carlos Compagnucci and Assistant First Team Coach Xavier Tamarit have also parted company with the club.
‘We would like to place on record our thanks to Mauricio, Carlos and Xavier for their efforts during their time with Southampton, and wish them well for the future.
‘The club will look to appoint a new management team as soon as possible, with the search for a replacement already underway.’
0.93 – Only Harry Redknapp (0.91) and Steve Wigley (0.64) have a worse points-per-game record among Southampton managers in the Premier League than Mauricio Pellegrino. Casualty.
— OptaJoe (@OptaJoe) March 12, 2018
Speaking on BBC Radio 5 live, former Southampton star Francis Benali said: ‘It is something that has been bubbling away and has been talked about by Southampton fans for a while.
‘I think Pellegrino has had a good opportunity to turn things around. It is late in the season, and he perhaps should have gone some time back but Southampton have stuck with him.
‘There have been a few must-win games that Southampton have not won. I think the heavy defeat against Newcastle and the manner of it has sealed his fate.
‘They are thinking hopefully it’s not too late for someone to come in and turn things around.’