WASHINGTON, Dec. 17 — Although the chances are slim that President Nixon will recognize the Bengali rebels’ regime any time soon, as the Indian Government has done, legal experts in the State Department have already begun studying the process under which a new state wins acceptance in the international community and the steps Mr. Nixon might take if he wishes to grant recognition.

According to these experts, few written “rules” govern recognition. What guidelines do exist, they said, are little more than customs handed down over the centuries.

To begin with, they said, a new state seeking recognition as a sovereign country should satisfy several traditional criteria. Its central government should be able to demonstrate that it rules with the consent of its people, controls its territory and is capable of making contractual bargains, such as treaties and trade agreements, with other nations.

The experts conceded, however, that nations may be recognized as sovereign entities even if they do not meet all these criteria. They noted that South Vietnam is recognized by the United States and others as a sovereign nation even though the Saigon Government's hold on its people is far from complete; and that the United States continues to deal with the Greek military junta, even though it has been heavily criticized by elements of the Greek population and by many expatriates.

“Even after you set forth the criteria, there is still plenty of room for discretion,” one State Department official said today. “And by that we mean room for political judgments. The world in general might recognize a nation as a nation, but this does not mean we have to.”

Political considerations that govern the recognition process are likely to play a heavy role in Mr. Nixon's decision on Bangladesh, State Department sources said.

According to these officials, the character of the present Bangladesh government — regarded here as secessionist—and the manner of its rise to power—military conquest, with help from a third party, India —will be less influential in the President's mind than broader political considerations involving Pakistan, with which the United States maintains treaty relationships, and Communist China, an ally of Pakistan. Mr. Nixon is likely to defer any action until the Pakistanis act on the matter.

This pattern would be consistent with United States diplomacy in the past. The United States has tended to defer recognition of new Latin American governments, for example, until other members of the Organization of American States have opened formal diplomatic contact.

“As a practical matter,” one State Department expert said, “what counts is how many people deal with you. There are many indices of nationhood. Recognition by various countries is one, treaty relationships another, trade relationships a third. You just try to build these up.”