The comma is one of the most common delimiters, as suggested by Comma Separated Value. However, there are other delimiter options as well. This often depends on regional variations in the use of the comma. For example, in Europe, the default delimiter is most often a semicolon, since commas are used to separate decimal fractions (e.g.in currency).
The most common delimiters are:
semicolon
tab
pipe (vertical bar)
space
The presence of a comma within values or text is bound to cause an error if the comma is used as a delimiter at the same time. Sometimes, when no other substitute can be used, the text has to be enclosed in quotation marks, then the spreadsheet or other program, where you import data, will ignore the comma as a delimiter, but that's a different topic for discussion. In any case, if you can't use a comma as a unique delimiter, choose something else.
To summarize, a delimiter in CSV is a character (comma or another) that separates the data in your file into separate values. Or in another way of saying and going back to tabular data, it divides rows in your CSV into columns.