For complete e-books (and "real" books at a library), I'd use:
emprunter for check out/borrow (see rubric "Pret")
retourner or rendre for check in/return (see rubric "Reservation et retour")
EDITED TO ADD THE FOLLOWING DISCUSSION OF THE IMPORTANCE OF CONTEXT (in this or any context):
Using “emprunter” and “retourner”/”rendre” in the context you describe (i.e., the collaboration and document management field) would, of course, require giving them the same “special meanings” that “check out” and “check in” in English have been given in that field, but no more so than in English.
Context-neutral uses of “emprunter” in French and “check out” in English imply "to borrow (the borrowed thing) for permitted/acceptable purposes," just as context-neutral uses of “retourner/rendre” in French and “check in” in English imply "to return (the borrowed thing) in an acceptable condition/form."
In the context of borrowing and returning a library book, in English and in French, the permissible/acceptable use or purpose is generally limited to “reading it” and the acceptable condition/form for its return is “as you received it, normal wear and tear excepted.”
However, in the context you mention, acceptable/permissible uses while checked-out apparently include working on/editing the borrowed document and saving any edits made to it; and an acceptable condition/form for its return would include “as you last saved it.” (I imagine that, just as in the library context, damaging/corrupting/destroying the borrowed file would not be a permitted/acceptable “use” of it, nor would a damaged/corrupted/destroyed condition or form be an acceptable condition/form for its return.)
If Anglophones (who so often require different words for different contexts [see “like/love,” for one example]), are capable of accurately understanding the “special meanings” of “check out” and “check in” in this context, I have full confidence that Francophones (who are so very talented at immediately capturing the meaning of words depending on their context [see “aimer” for one of many examples]), will likewise accurately understand the “special meanings” of “emprunter” and “retourner/rendre” when encountering them in this context.
(And besides, it would seem that English-speaking Francophones (i.e., those who know that "check out" and "check in" have other, "non-special" meanings than those used in this particular field AND who already use these English phrases in the field ) would appreciate (if not jump at) the opportunity to replace them with equivalent words, with equivalent "special meanings," from their own language.)