I have never found the term "bosser" in the mouth of anyone as used to mean "What's up?" (Quoi de neuf ?"), nor have I found that it could have any other meaning than "travailler".
However, the quick research I just made yields the following surprising result: it could also mean nowadays "Ça marche ?", ""Ça va ?" (in the sense of "Does it work?", "Is it working?"). This follows from the article to be consulted here. There is no doubt that the usual meaning is not relevant as there is a comma after "ça bosse", which shows that "les maths" can't be an object but only a postponed subject used redundantly ("bosser qqc" means "to work on sth"). In other words that could amount to saying "Are mathematics doing the trick?", "Is mathematics working?" or possibly "Is your/the work in mathemetics going well?", there being possibly at the same time, whichever is the right possibility or whichever other one is right, a play on words in reference to the expression "avoir la bosse des mathematics" (possibly also, empty of real meaning, I see none); this expression is used for other subjects than mathematics (bosse du commerce, du calcul mental, du dessin, etc.)
It becomes then more acceptable to suppose that the possibility of applying this to a nondescript situation should exist and even that this usage would be the initial expresssion that lead to what would be the extension examined above. This would mean however "Ça marche (bien) ?" or "Ça va (bien) ?" but it is unlikely to be "What's up ?". Nevertheless, given the context, I can't understand how one could think about work of any sort, that does not make sense.