Mathematics is a plural as it covers lots of different topics, arithmetic, calculus, trigonometry, statistics and so on. Historically in printing when shortening long words the trend would have been to use an apostrophe to denote removed characters (from within the middle of the word) so we get to Math’s. with a full stop ending. People would originally read the word in full but laziness and familiarity with the contraction eventually lead to the abbreviation being spoken as written. And then the apostrophe gets lost.