Development of a server for mobile clients. Development of a server for mobile clients Development of a server part of a mobile application

The development of the server side of a client-server application begins with the design of the architecture. A lot depends on the architecture: from the extensibility of the application to its performance and ease of support / maintenance.

First of all, you need to determine how the data will be placed on the server and how the requests coming from the client will be processed. You also need to consider the organization of server data caching.

It is necessary to determine the data exchange protocols and data transfer formats.

API (application programming interface) is an application programming interface. In more understandable language, this is a set of requests to the server, which the latter understands and can give the correct answer. The API defines the functionality of the server-side logic, while the API allows you to abstract away from how exactly that functionality is implemented. In other words, the API is a necessary part of the overall client / server infrastructure.

Compare JSON and XML. Provide an example of protocols depending on the type of application.

Multithreading

One of the key aspects in modern programming is multithreading. With the help of multithreading, we can allocate several threads in an application that will perform various tasks at the same time.

Multithreading is a property of a platform (for example, an operating system, a virtual machine, etc.) or an application that a process spawned in an operating system can consist of several threads running "in parallel", that is, without a prescribed order in time.

The essence of multithreading is quasi-multitasking at the level of one executable process, that is, all threads are executed in the address space of the process. In addition, all threads in a process have not only a common address space, but also common file descriptors. A running process has at least one (main) thread.

Multithreading (as a programming doctrine) should not be confused with either multitasking or multiprocessing, despite the fact that operating systems that implement multitasking tend to implement multithreading as well.

The advantages of multithreading in programming include the following:

Simplification of the program in some cases due to the use of a common address space.

Less time spent on creating a stream relative to the process.

Increase of process performance due to parallelization of processor computations and I / O operations.

Flow(threаd) is a managed unit of executable code. In a thread-based multitasking environment, all running processes must have a main thread, but there can be more. This means that multiple tasks can be performed asynchronously in one program. For example, editing text in a text editor while printing, since these two tasks are performed in different threads.

On a conventional processor, flow control is handled by the operating system. A thread is executed until a hardware interrupt, a system call, or the time allotted for it by the operating system expires. After that, the processor switches to the operating system code, which saves the state of the thread (its context) or switches to the state of another thread, which is also allocated time for execution. With such multithreading, a fairly large number of processor cycles are spent on operating system code that switches contexts. If thread support is implemented in hardware, then the processor itself will be able to switch between threads, and in the ideal case, execute several threads simultaneously for each clock cycle.

- Temporary multithreading (one thread)

- Simultaneous multithreading (multiple threads at the same time)

Multithreading, as a widespread programming and code execution model, allows multiple threads to execute within a single process. These threads of execution share the resources of the process, but they can run independently. The multithreaded programming model provides developers with a convenient abstraction for parallel execution. However, perhaps the most interesting application of the technology is when it is applied to a single process, which allows its parallel execution on a multiprocessor system.

This advantage of a multithreaded program allows it to run faster on computer systems that have multiple processors, a processor with multiple cores, or a cluster of machines - because program threads naturally lend themselves to truly parallel execution of processes. In this case, the programmer needs to be very careful to avoid race conditions and other unintuitive behavior. In order to properly manipulate data, threads of execution must frequently go through the rendezvous procedure in order to process the data in the correct order. Threads of execution may also need mutexes (which are often implemented using semaphores) to prevent shared data from being modified or read at the same time during the update process. Careless use of such primitives can lead to a dead end.

Another use of multithreading, even for uniprocessor systems, is the ability for an application to be responsive to input. In single-threaded programs, if the main thread is blocked by a long running task, the entire application can be frozen. By moving such time-consuming tasks to a worker thread that runs in parallel with the main thread, it becomes possible for applications to continue to respond to user input while tasks are running in the background. On the other hand, in most cases, multithreading is not the only way to preserve the sensitivity of a program. The same can be achieved via asynchronous I / O or signals on UNIX.

In total, there are two types of multitasking: based on processes and based on streams... The difference between process-based and thread-based multitasking boils down to the following: process-based multitasking is organized for parallel execution of programs, and thread-based multitasking is for parallel execution of separate parts of the same program.

In total, two types of streams are distinguished:

Foreground threads or foreground. By default, every thread created through the Thread.Start () method automatically becomes the foreground thread. These types of threads provide protection of the current application from termination. The CLR will not stop the application until all foreground threads have finished.

Background threads This type of thread is also called daemon thread, and is interpreted by the CLR as extensible paths of execution that can be ignored at any time. Thus, if all foreground threads are terminated, then all background threads are automatically destroyed when the application domain is unloaded. To create background threads, set the IsBackground property to true.

Tell about the states of threads: running, suspended, running, but waiting for something.

Thread synchronization problem and shared resources.

Interaction of threads

In a multithreaded environment, problems often arise related to the use of the same data or devices by parallel executing threads. To solve such problems, such methods of thread interaction as mutexes, semaphores, critical sections and events are used.

Mutex Is a synchronization object that is set to a special signaling state when not busy with any thread. Only one thread owns this object at any given time, hence the name of such objects (from the English mutually exclusive access) - simultaneous access to a shared resource is excluded. After all the necessary actions are taken, the mutex is released, giving other threads access to the shared resource. An object can support recursive capture a second time by the same thread, incrementing the counter without blocking the thread, and then requiring multiple deallocations. This is, for example, the critical section in Win32. However, there are some implementations that do not support this and cause the thread to deadlock when attempting a recursive capture. This is FAST_MUTEX in the Windows kernel.

Semaphores are available resources that can be acquired by multiple threads at the same time until the resource pool is empty. Then additional threads must wait until the required amount of resources is available again. Semaphores are very efficient because they allow concurrent access to resources.

Developments... An object that stores 1 bit of information "signaled or not", over which operations "signal", "reset to non-signaled state" and "wait" are defined. Waiting on a signaled event is the absence of an operation with the immediate continuation of the thread execution. Waiting on an unselected event causes the thread to be suspended until another thread (or the second phase of the interrupt handler in the OS kernel) signals the event. It is possible to wait for several events in the "any" or "all" modes. It is also possible to create an event that is automatically reset to a non-signaled state after waking up the first - and only - waiting thread (such an object is used as the basis for the implementation of the "critical section" object). They are actively used in MS Windows, both in user mode and in kernel mode. There is a similar object in the Linux kernel called kwait_queue.

Critical sections provide synchronization similar to mutexes, except that objects representing critical sections are available within the same process. Events, mutexes, and semaphores can also be used in a single-process application, but critical section implementations in some operating systems (such as Windows NT) provide a faster and more efficient mutually exclusive synchronization mechanism - get and release operations on the critical section are optimized for the case of a single thread (no contention) in order to avoid any system calls leading to the OS kernel. Like mutexes, an object representing a critical section can only be used by one thread at a time, which makes them extremely useful for delimiting access to shared resources.

Conditional variables(condvars). They are similar to events, but they are not objects that occupy memory - only the address of a variable is used, the concept of "contents of a variable" does not exist, the address of an arbitrary object can be used as a conditional variable. Unlike events, setting a condition variable to a signaled state has no consequences if there are currently no threads waiting on the variable. Setting an event in a similar case entails storing the "signaled" state inside the event itself, after which the next threads wishing to wait for the event continue execution immediately without stopping. To make full use of such an object, it is also necessary to release the mutex and wait for the conditional variable atomically. They are widely used in UNIX-like operating systems. Discussions about the advantages and disadvantages of events and conditional variables are a prominent part of discussions about the advantages and disadvantages of Windows and UNIX.

I / O completion port(IO completion port, IOCP). The "queue" object implemented in the OS kernel and accessible through system calls with the operations "put the structure in the tail of the queue" and "take the next structure from the head of the queue" - the last call pauses the execution of the thread if the queue is empty, and until another thread will not make the call to "put". The most important feature of IOCP is that structures can be placed in it not only by an explicit system call from user mode, but also implicitly inside the OS kernel as a result of an asynchronous I / O operation completed on one of the file descriptors. To achieve this effect, you must use the bind file descriptor to IOCP system call. In this case, the structure placed in the queue contains the error code of the I / O operation, and also, if this operation is successful, the number of bytes actually entered or output. The completion port implementation also limits the number of threads that run on a single processor / core after a structure is queued. The object is specific to MS Windows, and allows processing incoming connection requests and data chunks in server software in an architecture where the number of threads can be less than the number of clients (there is no requirement to create a separate thread with resource consumption for each new client).

Pool of threads

Tell about the thread pool

Much of the modern applications for mobile platforms(iOS, Android, etc.) works in tandem with the server. An application with outdated data loses its usefulness. Therefore, it is important to ensure that data from the server to the device is kept up to date. This applies to offline applications that should work without the Internet. For completely online applications that do not work (or are useless) without the Internet (for example, Foursquare, Facebook) there are specificities that are beyond the scope of this article.

Using the example of one of our offline applications, I will tell you what approaches we used to synchronize data. In the first versions we have developed simple algorithms and, in the future, with experience, we improved them. A similar sequence is presented in the article - from simple obvious practices to more complex ones.

It should be clarified that the article deals with data transfer only in one direction: from the server to the device. Here the server is the data source.

General provisions for all approaches

For example, we will consider passing a “dishes” directory to a device. We will assume that the device makes a request for the url “/ service / dishes / update”, the exchange is carried out via the http protocol in the JSON format ( www.json.org). The server has a table “dishes” with the following fields: id (record identifier), name (name of the dish), updated (the moment the dish is updated, it is better to do timezone support right away, “YYYY-MM-DDThh: mm: ssTZD”, for example, “1997 -07-16T19: 20: 30 + 01: 00 ”), is_deleted (sign of deleted record).

Remark regarding the presence of the last field. By default, its value is 0. In an application where entities are synchronized between the client and the server, it is not recommended to physically delete data from the server (to avoid errors). Therefore, is_deleted = 1 is set for deleted dishes. When an entity with is_deleted = 1 arrives at the device, it is deleted from the device.

With any approach, which will be considered below, the server returns an array of objects to JSON devices (may be empty):

[
(id: , name: , updated: , isDeleted: },…
]

Server response example:

[
(id: 5625, name: "Bread", updated: "2013-01-06 06:23:12", isDeleted: 0),
(id: 23, name: "Cooked semolina", updated: "2013-02-01 14:44:21", isDeleted: 0), (

name: "Fish-soup",

updated: "2013-08-02 07:05:19",

Principles of updating data on a device

  1. If an element that is on the device came and isDeleted = 0, then it is updated
  2. If an element has arrived that is not on the device, and isDeleted = 0, then it is added
  3. If an element that is on the device came and isDeleted = 1, then it is deleted
  4. If an element has arrived that is not on the device, and isDeleted = 1, then nothing is done

Approach 1: Everything is always in sync

This is the easiest method. The device requests a list of dishes from the server and the server sends the entire list. Every time the whole list comes. Not sorted.

Example request: null, or “()”

Advantages:

  • the logic on the server is simple - we always give everything
  • the logic on the device is simple - we always overwrite everything

Disadvantages:

  • if you request the list often (every 10 minutes), then there will be a lot of Internet traffic
  • if the list is requested rarely (once a day), then the relevance of the data will be violated

Application area:

  • for low traffic applications
  • transmission of very rarely changing data (list of cities, categories)
  • transfer of application settings
  • at the beginning of the project for the very first prototype of a mobile application

Approach 2: only sync the updated

The device requests a list of dishes, updated from the previous sync. The list comes sorted by “updated” in ascending order (optional, but convenient). The device stores the value “updated” for the most recently sent dish and, on the next request, sends it to the server in the “lastUpdated” parameter. The server sends a list of dishes that are newer than “lastUpdated” (updated> lastUpdated). At the first request to the server “lastUpdated” = null.

Example request: (lastUpdated: “2013-01-01 00:00:00”)

In the diagram: “last_updated” is the value that is stored on the device. Usually, a separate table is created on the device to store these “last_updated” values ​​for each entity (dish, city, organization, etc.)

This approach is suitable for synchronizing simple linear lists that have the same arrival rules for all devices. For more selective synchronization, see “Approach 5: Synchronize with the knowledge of what is already on the device”.

This approach usually covers most needs. Only new data comes to the device, you can synchronize at least every minute - the traffic will be small. However, there are problems associated with the limitations of mobile devices. These are memory and processor.

Approach 3: Synchronize in batches

Mobile devices are low on RAM. If there are 3000 dishes in the directory, then parsing a large json string from the server into objects on the device may cause a lack of memory. In this case, the application will either crash or not save these 3000 dishes. But even if the device was able to digest such a string, then the application performance at the moments of synchronization in the background will be low (interface lags, not smooth scrolling, etc.) Therefore, it is necessary to request the list in smaller portions.

To do this, the device passes one more parameter (“amount”), which determines the size of the portion. The list must be sent sorted by the "updated" field in ascending order. The device, similar to the previous approach, remembers the “updated” value of the last sent entity and passes it to the “lastUpdated” field. If the server has sent exactly the same number of entities, then the device continues synchronizing and makes a request again, but with the updated “lastUpdated”. If the server has sent fewer entities, this means that it does not have more new data, and the synchronization ends.

In the diagram: “last_updated” and “amount” are the values ​​that are stored in mobile application... “Last_item” - the last entity (dish) sent from the server. It is newer than this value that the next list will be requested.

Example request: (lastUpdated: “2013-01-01 00:00:00”, amount: 100)

Advantages:

  • The device receives as much data as it is able to process at a time. Serving size is determined by practical tests. Simple entities can be synchronized 1000 at a time. But it also happens that entities with a large number of fields and complex storage processing logic are synchronized normally no more than 5 pieces.

Disadvantages:

  • If there are 250 dishes with the same updated, then with amount = 100, the last 150 will not be sent to devices. This situation is quite real and is described in the following approach.

Approach 4: Correct batch timing

In the previous approach, it is possible that if there are 250 dishes in the table with the same “updated” (for example, “2013-01-10 12:34:56”) and the portion size is 100, then only the first 100 records will come. The remaining 150 will be hard-clipped (updated> lastUpdated). Why is this going to happen? When the first 100 records are requested, lastUpdated will be set to “2013-01-10 12:34:56” and the next request will have the condition (updated> “2013-01-10 12:34:56”). Even softening the condition (updated> = “2013-01-10 12:34:56”) will not help, because the device will then endlessly request the first 100 records.

The situation with the same "updated" is not so rare. For example, when importing data from a text file, the “updated” field was set to NOW (). It can take less than a second to import a file with thousands of lines. It may also happen that the entire directory will have the same "updated".

To fix this, you need to use some dish field that would be unique at least within one moment (“updated”). The “id” field is unique throughout the entire table, so you should additionally use it in synchronization.

So, the implementation of this approach looks like this. The server returns the list sorted by “updated” and “id”, and devices request data using “lastUpdated” and the new parameter “lastId“. At the server, the selection condition is more complicated: ((updated> lastUpdated) OR (updated = lastUpdated and id> lastId)).

In the diagram: “last_updated”, “last_id” and “amount” are the values ​​that are stored in the mobile application. “Last_item” - the last entity (dish) sent from the server. It is newer than this value that the next list will be requested.

Approach 5: Synchronize with knowledge of what is already on the device

Previous approaches do not take into account the fact that the server does not really know how successfully the data was saved on the device. The device could simply not save some of the data due to unexplained errors. Therefore, it would be nice to receive confirmation from the device that all (or not all) dishes have been preserved.

In addition, the user of the application can configure the application in such a way that he only needs a part of the data. For example, a user wants to synchronize dishes from only 2 cities out of 10. This cannot be achieved with the synchronizations described above.

The idea behind the approach is as follows. The server stores (in a separate table “stored_item_list”) information about what dishes are on the device. It could just be a list of “id - updated” pairs. This table contains all lists of “id - updated” dish pairs for all devices.

The device sends information about the dishes available on the device (the list of “id - updated“ pairs) to the server along with a request for synchronization. When requested, the server checks what dishes should be on the device and what they are now. The difference is then sent to the device.

How does the server determine which dishes should be on the device? In the simplest case, the server makes a request that will return a list of “id - updated” pairs of all dishes (for example, SELECT id, updated FROM dishes). In the diagram, this is done by the “WhatShouldBeOnDeviceMethod ()” method. This is the disadvantage of the approach - the server has to calculate (sometimes making heavy sql queries) what should be on the device.

How does the server determine which dishes are on the device? It makes a query to the “stored_item_list” table for this device and gets a list of “id - updated” pairs.

By analyzing these two lists, the server decides what should be sent to the device and what should be deleted. In the diagram, this is “delta_item_list”. Therefore, the request does not contain “lastUpdated” and “lastId”, their task is performed by the pairs “id - updated”.

How does the server know about the dishes available on the device? In the request to the server, a new parameter “items” is added, which contains a list of the id of dishes that were sent to the device in the last synchronization (“device_last_stored_item_list”). Of course, you can send a list of the id of all the dishes that are on the device, and not complicate the algorithm. But if there are 3000 dishes on the device and they are all sent every time, then the traffic costs will be very high. In the vast majority of synchronizations, the “items” parameter will be empty.

The server must constantly update the “stored_item_list” with the data that came from the device in the “items” parameter.

You should implement a mechanism for clearing server data in stored_item_list. For example, after reinstalling an application on a device, the server will assume that the device is still up-to-date. Therefore, when installing the application, the device must somehow inform the server so that it clears the stored_item_list for this device. In our application, we send an additional parameter “clearCache” = 1 in this case.

Conclusion

A summary table of the characteristics of these approaches:

№ An approach Traffic volume(5 - large) Labor intensity of development(5 - high) Device memory usage(5 - high) Correctness of data on the device(5 - high) You can select a specific device
1 Everything is always synchronized 5 1 5 5 No
2 Only the updated 1 2 5 3 No
3 Synchronization in batches 1 3 1 3 No
4 Correct synchronization in batches 1 3 1 3 No
5 Synchronization with knowledge of what is already on the device 2 5 2 5 Yes

“Correctness of data on device” is the probability that the device contains all the data that was sent by the server. In the case of approaches # 1 and # 5, there is 100% certainty that the device has all the data it needs. In other cases, there is no such guarantee. This does not mean that other approaches cannot be used. It's just that if a part of the data is lost on the device, then it will not work to fix it from the server (and even more so to find out about it on the server side).

Perhaps, in the presence of unlimited Internet tariffs and free wifi, the problem of limiting traffic generated by a mobile application will become less relevant. But while you have to go to all sorts of tricks, come up with smarter approaches that can reduce network costs and increase application performance. It doesn't always work. Sometimes it is "the simpler, the better", depending on the situation. Hopefully, from this article, you can pick an approach that comes in handy.

There are surprisingly few descriptions of server synchronization on the Internet and mobile devices... Moreover, there are many applications that work according to this scheme. For those interested, a couple of links.

BACKUP

Why do you need backups on a mobile platform

Experts know how sometimes unreliable mobile applications on 1C are: errors can occur at any time, due to which the user base will simply collapse. At the same time, we are faced with the unreliability of the devices themselves: they can be broken, lost, they can be stolen, and users want to keep their data. And up to version 8.3.9, we did not have a platform mechanism for saving a backup.

Since users did not previously have a "save a copy" button, the developers of the Boss application had to make their own backups. How did we do it?

We save the data of the database in the form of XML.

It is advisable to offer the user several options for storing copies - first of all, it is convenient for customers, they can choose the best option for themselves: upload to the cloud, send to their mail, save on the device.

Thus, the developers additionally insure themselves. If something went wrong, and the mechanism for creating copies on Google Drive or Yandex Drive suddenly broke down, you can always tell the user that at the moment the developer is dealing with an error, but for now he can save the data in an alternative way. And users are satisfied because they can rest assured about their data.

Necessarily need to focus on cloud services, because if the device is lost or broken, and the user saved a copy on the same device, then the data will be lost.

Also we be sure to remind the user of the need to create backups.

How do I keep copies if the configuration changes?

When we talk about a mass solution, about an application that is constantly changing, developing and refining, we must take into account the behavior of customers. The user may want to restore a backup saved in an old version of the application, which did not have any details. And then the task arises: to read the data, then fill in the data according to the update logic from the old version of the application. How to do it? In addition to the data, save the data structure itself so that later you know how to read it.

There are several options for storing this data structure, including it can be stored in the configuration itself. That is, every time a new version is released, keep the metadata structure of the previous version in the layout in the configuration.

Do not forget that in a mobile application, the configuration should not grow just like that, we should value the place in it, we should make it as compact as possible. But the application is developing, and there will be a lot of such layouts, and over time they will become more and more.

Therefore, in the case of a mobile application, another way is preferable - save the metadata structure directly in the data file... At the output, we get such a file, where at first we store some auxiliary data - the configuration version, the configuration scheme, the sequence boundaries, and after that we write the user data itself in XML format. Moreover, in the "Auxiliary data" section of the file, you can also store other important data that, for some reason, could not be written into XML.

We take the data schema that we have saved to the file, and on its basis we build the XDTO package for reading the file. We create a similar object in the database, fill it in, perform refill processing when updating, and save the already finished object to the database.

Below in the picture you can see a hint on how to write the XDTO model of these configurations beautifully. The company that released the Boss application experimented with this, found several ways, but settled on this option of recording the metadata schema. When you open the data file itself, you can see the usual structured XML, readable, which lists all the application metadata.

// Write the configuration scheme ModelXDTO = FactoryXDTO.ExportModelsXDTO ("http://v8.1c.ru/8.1/data/enterprise/current-config"); XDTO Factory.WriteXML (UploadFile, XDTO Model); // Reading the configuration schema Model XDTO = Factory XDTO. Read XML (Read XML, Factory XDTO. Type ("http://v8.1c.ru/8.1/xdto", "Model")); UnloadFactory = New XDTOFactory (ModelXDTO);

To protect the user, it is imperative to ask him again whether he needs to restore the backup. Maybe he was just experimenting and clicking on all the buttons in the application :) And now his current data may be lost. Therefore, when performing potentially "dangerous" actions, we always specify whether he really wants this, and how it should be done. The user must be aware of their actions.

There must be a mechanism for creating backups when we talk about an autonomous solution, when the user has all data stored exclusively on a mobile device: the user can lose his device, and then the data will be lost. And, it would seem, if the application does not work autonomously, but is connected to a central server, then the user should not have such a problem, because if the device is lost, he will connect to the server, receive all his data from the server again, and everything will be ok.

However, users do not always use backups the way we expect them to :) They very often use them to simply "roll back" data back. This is really a very strange behavior, but users of mobile applications are too lazy to figure out where they could make a mistake when entering data, and they just roll back the data and re-enter the data for the current day. After analyzing the statistics of working with the Boss application, we realized that this is a normal practice and this user behavior is more common than we might have expected.

And if you use synchronization with other devices, then you have to handle it. There are several solutions here:

  • break the connection with the server, specifying that the data on it will remain the same, and the copy will be restored only on the user's device;
  • it is better for the user to let him restore a copy at once on all devices, having previously prescribed such mechanisms.

There is one more thing here. Until now, we saved the backups ourselves, controlled the entire process, and caught the user's actions right in the code when he pressed the "save a copy" button. All this can be processed later. In the 8.3.9 platform, it became possible to save backups exactly by means of the platform. And the user does this without our knowledge. If synchronization with a central database is used, then such a scenario must be handled. We must somehow find out on our server that the user has restored a previously saved copy and must give him some kind of solution. We cannot afford to have data out of sync.

EXCHANGE

When we talk about a private solution on a mobile platform, we usually have a customer who, for example, wants to use a mobile platform for their sales agents, and so that they exchange data with a central database. Everything is simple here: one database, several devices, you raise the server, set up communication with it. So the problem of exchange between devices is easy to solve.

But if we are talking about a mass application where there are many databases, each of which has a very large number of users, the situation becomes more complicated. Users downloaded the app from the market and they want to sync with each other. For example, a husband downloaded an application for accounting for personal finances, and now he wants his wife to connect too, and they work together in the same application. There are many users, the application is developing, growing, and there is a need for a large, large number of databases. How to organize all this? Users will not personally contact developers to create a separate database for them and enable synchronization. They want to push a button and make it work right away. At the same moment.

How to proceed? This is where the data sharing mechanism comes to the rescue. It allows you to organize a single database, where there is one common configuration, but at the same time, an unlimited number of user bases are stored within one common database.

The best part is that you can add users dynamically, programmatically, without our participation. In reality, users simply click on the "register on the server" button, and everything happens by itself: a personal database is created for him on the server, and he can immediately start working in it.

How to do it? The first and easiest solution is to write your own server base with this mechanism. When our company started making the Boss application and exchanges in it, in the first version we did just that: we wrote a server database with a data sharing mechanism. Everything worked, especially since there was nothing complicated - the base separator is a common props.

But then we realized that we were reinventing the wheel :) In fact, there is a ready-made solution, and it has already taken into account the points that we had not even thought about yet. This is 1C: Fresh.

The scalability of the service is thought out here: what to do when there will be a lot of data and databases, how to grow with all this. There is a point about creating backup copies of data areas: that is, we do not just backup one common database, we make copies of a specific user. Moreover, the mechanism there is such that copies are made only when they are really needed. If a user has not logged into the database for a week, then we do not make copies of him, because nothing has changed there. Another feature of Fresh is that the service implements a mechanism to reduce the load on the server, which is very important when you have a lot of databases.

In general, Fresh is something new and interesting for us. We are slowly trying to figure it out, but for the most part we are just happy with its work.

Data transfer. How to implement it for exchange between devices

The platform provides two mechanisms - SOAP and http services. There are nuances of how to access these services when the data sharing mechanism is involved. In particular, you need to add parameters that indicate the specific number of the area to which you are accessing, because the platform cannot determine which database to access by the username. In addition, one and the same user can work with several databases within a single database (see the picture).

As for services, the Boss app implements instant exchange: one user enters data, and the other receives it. Users of mobile applications are accustomed to the fact that everything happens instantly, so we thought about which service is better to use - SOAP or http. Connection speed played a key role. In http, the connection speed is much higher, and when connecting via SOAP, we get a description of the service, which is heavy and takes a long time to load. The platform has a way to store a description of the service, but due to the parameters we add dynamically, we cannot use WS references. In addition, access to http services is more convenient and flexible in our experience.

So, our goal is to implement the exchange in real time. That is, we try not to make the user have to go somewhere, click on a button, think about how relevant his data is, whether he should update it ... The data should always be relevant for users. They are so used to working in instant messengers - one sent the data, the other immediately received it. Everything happens instantly. The same applies to applications related to business: one seller has issued a sale, the other must immediately see the current situation without taking any action.

Therefore, the Boss app uses background jobs for exchanges. After each data is written to the database, a background job is started, which initiates the exchange. The first part is to send data to the server. Then other devices need to know that there is new data. For this we use PUSH notifications. This scheme is already working and it works fast enough.

But we wanted it even faster, because we work in real time and we usually have little data. We have small XML, but at the same time we send a message with this data from the first device to the server, the server sends PUSH to another device, and then the second device, after receiving PUSH, initiates an exchange from its side, addresses the server and requests data, receives this data and then sends a response that the data has been received. This is a long time, but the data itself was very small.

We thought about how this process can be accelerated.

To do this, we figured out what PUSH contains, how it can still be used. It turned out that PUSH contains fields such as data and text. The iOS and Android documentation contains restrictions on the size of PUSH messages, but this seemed not enough to us and we wanted to figure it out empirically. And we checked that for iOS the sum of valid characters is 981 characters, and for Android it is 3832 characters. In the latter case, it is quite possible to use the restriction; one or several base objects can be crammed into such a volume. And then the developers of the company changed the scheme a little. When there is not much data, we send it from one device, receive it on the server, pack it in PUSH there and send it directly to another device in it. The scheme has become shorter, and the exchange has become even faster :)

An important point of using PUSH is not to annoy users.

It is very easy to get rid of this situation: just do not send a lot of PUSH messages to the user :) If he is working in the application now, you can send a lot of messages. When the platform is running, the user does not see PUSH, everything happens automatically. But when the application is closed, the client has a lot of unread messages. Therefore, in no case should the next PUSH be sent until a response is received from the device that the application is running, active, and the previous PUSH has already been processed.

Another nuance of the exchange is work via the web. We need to make the most of asynchrony. You cannot work as usual - write the code - call the function - wait for it to execute - get the answer - and everything is ok. If you work over the web, you will still face certain limitations, for example, unstable Internet, triggered timeouts when performing long operations. Therefore, it is necessary to think over the architecture in advance.

Let's look at an example of registering a device, what happens in an application when a user wants to register. He keeps records for a while, he entered a lot of data, but then he wants the seller to work with this database too. The user clicks on the "register" button. At first everything was very simple: they took his data, recorded it on the server, and, please, you can work and connect users. But then we ran into a situation where for some users the databases on the device by the time of registration had already grown greatly. And this scheme did not work any more, tk. while the entire database was being recorded on the server, the connection timeout was triggered or the Internet was simply cut off. Therefore, we replaced one synchronous call with many short ones. Now the data is being shared rather than being transmitted all at once. We do not wait in any way for the server to process and record data. We sent data, received a response that the data was received, closed the connection. Periodically, you need to poll the server, what is happening there and how, and in the meantime, a background job is running on the server, which records the received data. This way we get a lot of server calls, but we have a guarantee that everything will go well. And neither timeouts nor Internet instability will prevent you from uploading all the data to the server.

UPDATES

Exchange between devices with different versions of the application

Since we are talking about a mass application that is released to the markets, we must take into account some of the features of the update and data exchange process.

If you released an application for one enterprise and decided to update it, then usually you just give a command for all employees to install the new application together. This cannot be done with users who downloaded the app from the market. You can't tell them what to do at all. For example, they are working in an application and do not want to update it either now or ever. They do not have auto-update, so it is quite common situation when several devices are connected to the central base, and they are all with different versions. Another reason for this phenomenon is the publication time in the markets: it is different for iOS and Android. We often implement key things, for example, fix critical bugs, and do not want to wait for iOS to check the new version for two weeks, we want at least only for Android, but we want to release the update right now.

We have no right to command users. If they want, they are updated, and if not, they do nothing. The picture shows the ratio of Boss app installs by versions in GooglePlay, as well as statistics from our server - the real ratio of app versions that are installed on devices that have exchanged data with the server during the last week. This is the set to work with. These are different versions and different metadata. And we need to organize a normal exchange at the same time :)

The developers are faced with the following tasks:

  • All this needs to work. Users shouldn't feel discomfort that they forgot to upgrade. They shouldn't notice it at all. Updated - it's better, well and good.
  • We must ensure the safety of the data. For example, one user has a directory and a new props, while another does not yet. At the same time, if a user who has no new details changes something on his device, then on other devices the data should not be lost.
  • We need to ensure that the data is updated when we upgrade to a new version. When the user decides that he is ready to update, he should automatically have all the new information that he did not have just because he had an old version.

How did we do it?

1. We use 2 exchange plans on the server. The first is for sharing between devices, and the second is for updates. For example, we sent a manual to a user, but he does not have units of measurement, that is, incomplete data. We must remember this. And when he is updated, we must send him all the information that he did not have. This is what the second exchange plan is for.

2. To write and read objects, we use the same mechanism that is used for backups, that is, we save the version of the metadata. In this case, we are working with the server, and we can afford to add anything we want directly to the configuration, so we simply add metadata schemas to the configuration in the form of layouts as the application develops.

How to monitor massive errors during exchange and on the server

First, you need to control the availability of the server itself. This happens to servers - they fall. We did not invent anything special for monitoring, but simply found a bot in the telegram that screams if something is wrong. He checks the server's performance every minute, and if the server is suddenly unavailable, he starts screaming, the admins see it and bring up the server.

We also collect the error log from the log. Also, nothing supernatural - we just collect a log of errors every three hours, send them to the mail, periodically review them. This helps to see common problems and some kind of exceptional situations. It's not hard to read your mail, track down and quickly fix errors. But this allows you to quickly identify and solve problems that can grow with the growth of databases.

Another important point - be sure to give the user the opportunity to "complain". It improves our status in their eyes and saves us. There are users, as we call them, "hysterics" who, at the slightest mistake, begin to send us a bunch of messages by mail that nothing works, the database does not load, everything is terribly bad. But sometimes they really save us, because sometimes they find such bugs that others miraculously haven't found, serious bugs.

The user cannot be frightened. Not scary messages, nothing else. They need to explain everything beautifully and offer to complain. And we promise to solve everything. Then the users are happy, because they see that they are taken care of, and immediately believe that they will be helped :)

This article was written based on the results of the report read at the INFOSTART EVENT 2016 DEVELOPER conference. More articles can be read.

In 2020, we invite everyone to take part in 7 regional meetups, as well as the anniversary INFOSTART EVENT 2020 in Moscow.

The downside of mobile clients is the server.

Additional requirements depend on the specifics of the application:
server scalability - for SaaS, social applications, where, ideally, a large flow of visitors is expected, this condition is mandatory. For business applications where there are restrictions on the number of users or the number is predicted, this property is not required;
interactivity: a number of applications need to be provided with a notification mechanism - inform the application (user) about the occurrence of certain events, send a message to the user. This property should be possessed, for example, by an exchange system or an automatic taxi dispatcher.
open API: it is assumed that third-party developers can use the functionality of the system through a documented protocol. After all, a client can be either a mobile or an external server application.
other requirements ...

Command
The composition of the project team for the development of the system can ideally be as follows:
project manager: manages, controls the project, interacts directly with the customer;
server application developer: develops a business logic server, database, network protocol;
administrator application developer: develops a Web application, a user interface for configuring and managing a server application;
client application developer for Android;
iOS client application developer;
client application developer for ...
tester: tests the admin application and client applications.

The attentive reader will notice that if you write a server application with a graphical interface, for example, in HTML5, you can save money. In this case, the development of client applications is not required - the user interface is provided by the browser. This article does not cover such a case, it is about the development of "native" (native) applications for mobile devices.

I have worked in a team with a full staff, but I will be realistic - not always the human resources and budget allows you to assemble such a team. And sometimes the roles have to be combined: project manager + server application developer, client application developer + tester.

Technologies, tools, libraries
To develop a server for mobile clients, I usually use the following stack of "free" technologies:
Apache Tomcat is a servlet container;
MySQL - DBMS;
Subversion is a version control system;
Maven - a framework for automating project builds;
JUnit - will provide;
Apache Log4j - logging library;
Jenkins - continuous integration system;
Hibernate - ORM (settings, configuration in properties, xml files and annotations);
hibernate-generic-dao - implementation of DAO from Google, implements basic methods for working with database data, simplifies the implementation of filtering and sorting in methods;
- implementation of authentication and authorization (security), container of services and beans (configuration in xml files and in annotations), we also use when creating tests.

Depending on the specifics of the system and the requirements for it, I use one of 2 options for implementing the data exchange protocol.
When cross-platform, performance, simplicity, efficiency, scalability, open API are required, then I take Jersey - the implementation of REST Web services (RESTful Web services). This library allows you to use JSON and / or XML data serialization. REST configuration is done through annotations. For exchange with mobile devices, the JSON format was taken due to the fact that it has a simpler implementation on the client side (for this reason, we do not use “classic” Web services), less traffic is generated. Jersey allows you to tune in to the most appropriate JSON “look”.
Otherwise, if you need cross-platform, high performance, simplicity, efficiency, interactivity, then I take
Apache MINA is a framework for building network applications,
Google protobuf is a structured data encoding and decoding library. The data structure is determined by the * .proto header files, the compiler generates Java classes from them (there is also the possibility of generation for other programming languages: C ++, Objective-C, etc., which provides the cross-platform property);
java.util.concurrent - we use the standard package.
This option can be scaled, but it needs to be laid down at the design stage at the architecture level, taking into account the business logic.

Let us consider a hypothetical task using the example of choosing technologies for a real SaaS service - “Auction of services“ Auknem ”, which allows people to place an order for the performance of the required services or works, and organizations, in turn, leave their proposals for them. We take all the basic requirements by default. Due to the fact that registration in this system is free and free, it is unambiguously required to add scalability to them. What about interactivity? It would be great to inform contractors (performers) about the creation of new orders, and to inform customers about the proposals received at the same moment in the application, and not just by e-mail. On the basis of this, we take for the implementation of Apache MINA, Google protobuf. We look at the next property - open API. The service is publicly available, so let's assume that external developers might be interested in integrating with it. Wait a minute! Not so simple. The Apache MINA-based protocol is quite implementation dependent and integration without knowing the nuances is by no means transparent. In such a situation, you will have to weigh which factor is more important and make a choice.

Conclusion
I would be interested to know what technologies and libraries did you use when developing a server for mobile devices or similar systems? Everything changes, nothing lasts forever, at each level there are alternatives with their own advantages and disadvantages: MySQL -

Development of the server side of the application

Introduction

An internet presence has become a necessity for today's businesses. Without this, it is impossible to build a full-fledged interaction with customers. Often, to solve such a problem, they resort to creating client-server applications. Each of them consists of a client side and a Back-end. The last term refers to the server side of the application. If in the future you need to independently change the content of the mobile program, then the Back-end should be created with a particularly high quality. Appomart guarantees the fulfillment of the assigned tasks in accordance with the requirements. Therefore, when ordering the creation of server applications, you can be sure of the proper result.

What is the Back-end for?

Development of client-server applications involves two parts. The first, Front-end, accepts requests from users. It is visible from the screens of customers' mobile devices. The second, the server application, processes the received requests and acts as an administrative panel. Databases, program logic are stored here. Without this, no client-server application will work. In fact, the Back-end is the heart of the program. This is the intelligence that is responsible for processing customer requests, the speed of the application. Therefore, it is important that the architecture of the server application is thought out to the smallest detail, so that even highly loaded services work smoothly and quickly.

How to choose a programming language?

During the preparation of the technical assignment (part of the working documentation for the project), the architect designs the database system and links, describes the objects and their properties, and also develops the necessary server methods (requests that will be "used" by mobile applications referring to the server).

The importance of documentation and abandoned projects

Appomart is often approached by customers who have been abandoned by other contractors for one reason or another. And we take someone else's, sometimes even incorrectly working project, carry out its audit and subsequent revision and support. In the process of studying the source code and materials received from the customer, we are faced with the fact that many developers deliberately do not document the server methods in order to bind the client to themselves, due to the incommensurable labor costs of transferring the project to support another developer, due to the lack of documentation for the server side. and sometimes just because of lack of professionalism. This fact, unfortunately, is not only sad but also widespread. The customer, in this case, needs to pay for the development of documentation for an existing project, as well as an audit of the source code, before it will be possible to judge the operability, convenience and expediency of project support. Appomart staff always maintain electronic documentation of back-end methods in a format supported by Postman and Swagger for future reference.

How to check the contractor before signing the contract?

We urge you to carefully choose a contractor, and focus not only on the tempting price, but also on the list of documents that you will receive with the project, as well as the conditions for transferring the source code, and the coverage of the code with comments, database schemas (be it Mongo DB or MySQL ). The key to success, as a rule, becomes competent working documentation, which clearly indicates the requirements for the materials transferred to you upon completion of each stage of work.

Development features

PHP for the server side

Creation of the server side of applications (not to be confused with servers as "hardware" or computers, since we are talking about the software part) requires specific professional skills and knowledge of the programming language that is used on the server side. If we look at examples of client-server applications, we can see that PHP is popular. It is the undisputed leader in server application development. More than half of the world's sites are written in this language in one configuration or another. PHP is easy to develop and maintain, and there are special frameworks to speed up PHP development.

Framework

Framework (software platform) - used to organize and increase the levels of abstraction, which makes the project more flexible and scalable. However, it should be understood that the framework must be chosen correctly, based on an in-depth analysis of the working documentation of the project, without which it is impossible to develop a quality product.

Delphi, JAVA, Python

There are other languages ​​that are used to create the Back-end. So, server applications created in the Delphi environment are widespread. With its help, the program receives improved debugging, it is also easy to create unique programs in the environment, visual creation is provided, which makes it possible to create a beautiful, understandable and convenient interface. Java server applications have also gained popularity. These are easily supplemented, easily executed on any platform, and have a decent level of security. Another popular language is Python. Server applications with its help are created quickly, simply, without serious costs.

Spreading

Creation of client-server applications is in demand in the corporate environment. Often such programs are used for workgroups or the creation of information systems within the enterprise. The overwhelming majority of mobile applications for maintaining communication with the client also have a similar architecture. The popularity is due to the fact that the use of server capabilities allows you to ensure control and integrity of the system, while reducing the load on the network.

We will create a client-server application for Android, iOS with high quality and on time

Turnkey development

Appomart programmers are experienced and qualified to handle tasks of a wide variety of levels. We are equally good at implementing social networks, high-load business projects, or the software part for small startups. If necessary, we will create the client part of the application running Android, iOS in accordance with the existing needs and requirements.

Back-end in Appomart

Our programmers work with a variety of technologies and do it equally well. In Appomart you can order a client-server application in Java, PHP and Node.JS. System requirements are analyzed for each project individually to ensure optimal program performance. Let's create a client-server application Java, PHP and Node.JS from scratch or take an existing one into support for improvements and updates. If you are interested in developing a new server part or supporting an existing one, leave a request to receive a detailed calculation of the cost of work and options for cooperation.

Fitting © 2021 Tech Look - Gadgets. Android device. Antiviruses. Security. Graphics. Internet. Mobile devices