You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-26 - 03:36

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot7s.osadl.org (updated Sun Apr 26, 2026 00:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
6643212135,1191sleep329076-21kworker/3:119:45:583
19779276414,729sleep019854-21/usr/sbin/munin22:11:030
262002483102,277sleep326346-21unixbench_singl20:46:123
14236243276,229sleep314287-21cpuspeed23:35:553
119252416112,229sleep312074-21unixbench_multi20:01:133
27459231340,89sleep30-21swapper/319:10:403
8804226762,109sleep38958-21unixbench_multi19:51:113
1244022659,218sleep312585-21latency_hist23:30:423
18149224623,205sleep30-21swapper/322:06:013
929223235,27sleep30-21swapper/319:26:113
7045223025,86sleep30-21swapper/321:30:553
7457222215,110sleep37486-21modprobe23:15:403
29311221640,84sleep30-21swapper/320:56:143
1568822149,17sleep30-21swapper/320:15:423
1568822149,17sleep30-21swapper/320:15:423
1610522077,186sleep313441-21snmpd22:00:403
1610522077,186sleep313441-21snmpd22:00:403
22067220526,162sleep122101-21awk20:35:401
22117220434,110sleep322165-21fschecks_count00:01:043
11767220076,27sleep30-21swapper/321:45:583
3070021978,129sleep330829-21grep00:26:123
2011219713,169sleep22021-21ntp_states22:56:242
12658218975,103sleep312708-21cpuspeed23:30:553
2514821867,87sleep30-21swapper/322:26:123
1012921778,81sleep30-21swapper/323:21:253
2107821687,146sleep321157-21cat20:31:013
30211216669,21sleep30-21swapper/319:16:093
25848216521,58sleep00-21swapper/020:46:050
31447216329,92sleep331239-21/usr/sbin/munin19:21:023
31447216329,92sleep331239-21/usr/sbin/munin19:21:023
28216216151,88sleep30-21swapper/319:11:003
29620216022,92sleep30-21swapper/322:41:113
1863921578,57sleep30-21swapper/323:50:423
31307215417,107sleep231337-21awk00:30:392
2860321537,36sleep30-21swapper/300:21:113
31544215260,78sleep331689-21date21:05:433
4642215110,60sleep30-21swapper/323:05:543
17090214614,91sleep317159-21latency_hist23:45:413
17090214614,91sleep317159-21latency_hist23:45:413
19484214260,17sleep30-21swapper/322:10:533
3142821380,10sleep125990-21cobalt_printf21:05:401
632621377,115sleep36362-21cut19:45:423
16809213415,104sleep115901-21/usr/sbin/munin20:16:131
16540213425,97sleep316662-21timerandwakeup20:16:113
17619213257,35sleep30-21swapper/320:21:043
13895213214,23sleep30-21swapper/321:51:113
898421319,106sleep38995-21uname21:36:043
898421319,106sleep38995-21uname21:36:043
23542213119,94sleep023669-21unixbench_multi22:21:120
13174213157,34sleep30-21swapper/320:06:093
1517213032,79sleep31536-21apt-get22:55:533
23141212724,17sleep30-21swapper/320:36:143
30823212516,91sleep030945-21cat21:01:110
28335212446,34sleep30-21swapper/322:36:143
28335212446,34sleep30-21swapper/322:36:143
27632212212,94sleep227698-21cut22:36:012
27632212212,94sleep227698-21cut22:36:012
19225212220,13sleep30-21swapper/320:25:483
9877212022,76sleep29895-21awk21:40:392
9877212022,76sleep29895-21awk21:40:392
2479212040,39sleep30-21swapper/319:31:123
30722211969,34sleep30-21swapper/322:45:593
30722211969,34sleep30-21swapper/322:45:593
22620211927,25sleep30-21swapper/322:20:433
4020211814,17sleep30-21swapper/321:20:573
24886211830,66sleep224949-21latency19:01:022
27613211410,89sleep327628-21vmstat00:16:143
13999211214,82sleep214136-21cat23:35:422
20470211023,17sleep30-21swapper/323:55:553
19489210926,68sleep30-21swapper/323:51:103
22660210812,80sleep222749-21/usr/sbin/munin20:36:052
32705210726,45sleep3316-21mailstats22:51:123
20086210720,63sleep220221-21grep20:27:172
345321045,19sleep30-21swapper/321:17:593
345321045,19sleep30-21swapper/321:17:593
20593210410,81sleep120680-21fschecks_time23:56:011
4711210311,77sleep20-21swapper/221:21:132
1151210319,66sleep01329-21sed22:55:420
16427210217,59sleep20-21swapper/223:41:122
16427210217,59sleep20-21swapper/223:41:122
12331210020,59sleep112406-21timerandwakeup21:46:121
2836529920,15sleep30-21swapper/320:55:413
2632929910,72sleep226352-21/usr/sbin/munin22:31:042
2632929910,72sleep226352-21/usr/sbin/munin22:31:042
14002997,14sleep30-21swapper/321:11:123
2955929810,71sleep229619-21/usr/sbin/munin22:41:102
1560829817,65sleep115673-21lockfile-remove20:15:401
1560829817,65sleep115673-21lockfile-remove20:15:401
869329623,48sleep18810-21grep23:16:471
1711429614,57sleep017271-21unixbench_singl22:01:130
1711429614,57sleep017271-21unixbench_singl22:01:130
3063129512,60sleep00-21swapper/019:16:140
228582959,28sleep30-21swapper/322:20:553
2510129316,61sleep025184-21mailstats19:01:080
248512927,72sleep00-21swapper/020:41:200
263762917,74sleep313441-21snmpd00:15:253
1502229113,63sleep015162-21sed20:11:110
1502229113,63sleep015162-21sed20:11:110
120822916,12sleep20-21swapper/221:46:052
2810229028,40sleep20-21swapper/200:20:412
899828923,50sleep09042-21ntpq21:36:050
899828923,50sleep09042-21ntpq21:36:050
2922328913,61sleep229310-21sort20:56:132
1452328914,61sleep014591-21/usr/sbin/munin21:55:400
1125428920,32sleep30-21swapper/321:43:273
2013328726,40sleep020171-40install23:55:390
438028614,53sleep04383-21/usr/sbin/munin19:36:140
309528613,55sleep00-21swapper/023:00:550
47522858,64sleep31-21systemd19:40:413
325822858,16sleep20-21swapper/221:06:142
282092858,42sleep20-21swapper/219:10:592
1747028513,33sleep20-21swapper/220:20:542
549628411,33sleep30-21swapper/321:25:543
2400728431,13sleep30-21swapper/320:40:593
1284228415,55sleep012888-21basename23:31:000
1111428418,34sleep10-21swapper/123:25:561
99122838,13sleep30-21swapper/319:56:043
72328316,48sleep1319-21/usr/sbin/munin19:26:051
550028313,54sleep25563-21/usr/sbin/munin21:25:572
3168628315,46sleep231574-21/usr/sbin/munin00:30:592
1703928322,44sleep117062-21apt23:45:391
1703928322,44sleep117062-21apt23:45:391
1639528321,17sleep30-21swapper/322:00:563
1639528321,17sleep30-21swapper/322:00:563
1020628317,48sleep010325-21/usr/sbin/munin23:21:270
2646928214,55sleep226507-21latency_hist00:15:412
854628113,34sleep30-21swapper/323:16:453
296962810,7sleep00-21swapper/022:41:110
280628123,37sleep12829-21basename23:00:391
1994928117,41sleep119985-21proc_pri22:11:081
193282817,55sleep00-21swapper/020:26:460
741528014,50sleep07446-21nfs_client21:31:040
682628014,51sleep26942-21ls23:11:112
2464828012,17sleep10-21swapper/122:26:021
208828031,26sleep22115-21awk21:15:392
319572787,31sleep30-21swapper/321:06:033
3126427815,40sleep231308-21/usr/sbin/munin19:20:572
3126427815,40sleep231308-21/usr/sbin/munin19:20:572
2751127812,51sleep127523-21munin-node19:10:421
235742787,34sleep30-21swapper/300:05:563
1830927715,45sleep218333-21netstat22:06:042
838527619,33sleep20-21swapper/219:51:022
83032767,16sleep00-21swapper/023:16:380
718627618,37sleep07287-21grep19:46:110
1414127613,48sleep114283-21latency_hist20:10:441
1054727614,44sleep210594-21users19:56:152
443227518,15sleep30-21swapper/323:05:423
317542758,56sleep00-21swapper/022:48:510
317542758,56sleep00-21swapper/022:48:510
270472755,12sleep30-21swapper/322:33:273
270472755,12sleep30-21swapper/322:33:273
246982755,58sleep20-21swapper/200:08:582
119027516,28sleep10-21swapper/121:11:061
1156127514,46sleep011568-21ntp_states23:26:070
848527418,28sleep00-21swapper/019:51:040
2629227417,22sleep00-21swapper/019:06:000
731227318,28sleep20-21swapper/221:31:022
373927223,22sleep10-21swapper/119:36:021
2328427216,34sleep022357-21/usr/sbin/munin20:36:150
1919827223,23sleep10-21swapper/122:10:391
1554227217,25sleep00-21swapper/023:40:410
357227122,25sleep20-21swapper/223:01:092
3008527120,23sleep20-21swapper/219:16:042
2752427119,36sleep227640-21latency_hist19:10:432
2639327123,29sleep026398-21modprobe00:15:390
1319227126,24sleep10-21swapper/123:31:091
636427017,36sleep06508-21latency_hist19:45:430
316992706,11sleep30-21swapper/300:31:003
1906027017,40sleep119104-21basename23:51:001
301326915,39sleep13100-21cat21:16:111
301326915,39sleep13100-21cat21:16:111
1431026913,42sleep213441-21snmpd20:10:492
1431026913,42sleep213441-21snmpd20:10:492
1136726918,24sleep00-21swapper/020:01:010
504626810,15sleep10-21swapper/123:06:051
2610126818,22sleep20-21swapper/220:46:112
2488026823,26sleep00-21swapper/000:10:420
2224126816,29sleep10-21swapper/100:01:061
1025526814,35sleep210315-21/usr/sbin/munin21:41:002
3252026714,37sleep132555-21idleruntime-cro19:25:401
3252026714,37sleep132555-21idleruntime-cro19:25:401
3121226723,27sleep20-21swapper/222:46:102
3121226723,27sleep20-21swapper/222:46:102
2850726713,20sleep10-21swapper/100:20:571
161032675,12sleep10-21swapper/122:00:381
161032675,12sleep10-21swapper/122:00:381
1259526722,20sleep10-21swapper/120:05:431
99502669,13sleep10-21swapper/123:21:191
311622665,49sleep00-21swapper/000:28:470
2682926617,32sleep126973-21latency_hist20:50:421
1779826623,21sleep20-21swapper/223:46:082
140402667,50sleep00-21swapper/020:10:310
753826515,28sleep20-21swapper/219:46:582
275726520,29sleep22775-21/usr/sbin/munin21:16:042
275726520,29sleep22775-21/usr/sbin/munin21:16:042
255522657,18sleep20-21swapper/222:29:142
1142526525,23sleep10-21swapper/121:45:391
1132226521,26sleep20-21swapper/220:00:592
41826421,28sleep2425-21df_inode19:25:582
323826415,26sleep10-21swapper/119:35:421
2751226413,32sleep10-21swapper/122:35:511
2751226413,32sleep10-21swapper/122:35:511
256492646,11sleep00-21swapper/022:30:290
225062646,45sleep10-21swapper/122:19:521
185352648,43sleep10-21swapper/123:49:591
149526424,20sleep00-21swapper/021:11:130
511426317,22sleep20-21swapper/219:41:002
2294226322,24sleep10-21swapper/120:36:121
1616326313,28sleep20-21swapper/220:16:022
1355826324,20sleep20-21swapper/220:06:172
3217426219,28sleep132243-21/usr/sbin/munin00:31:111
303762628,17sleep20-21swapper/221:01:012
2823026225,17sleep00-21swapper/019:11:010
2804726216,20sleep00-21swapper/022:36:110
2804726216,20sleep00-21swapper/022:36:110
1886126216,18sleep10-21swapper/120:25:421
392526110,14sleep10-21swapper/121:20:511
2679226118,22sleep00-21swapper/000:15:590
2088026114,31sleep220889-21diskstats20:30:562
1801826121,21sleep00-21swapper/022:05:580
1350126116,29sleep213535-21irqstats21:51:012
450526017,18sleep00-21swapper/021:21:110
40322605,17sleep10-21swapper/123:01:321
2735226013,32sleep127474-21timerwakeupswit00:16:121
2730126017,27sleep027429-21ls22:35:420
2730126017,27sleep027429-21ls22:35:420
2518526018,20sleep10-21swapper/119:01:091
2211526018,26sleep222116-21diskstats00:01:032
1961926016,21sleep10-21swapper/120:27:081
1760326017,27sleep00-21swapper/023:46:020
3200325918,26sleep00-21swapper/021:06:040
2820625910,37sleep013441-21snmpd20:54:290
272222596,12sleep10-21swapper/120:51:031
2663125915,25sleep10-21swapper/122:31:111
2663125915,25sleep10-21swapper/122:31:111
248622595,10sleep10-21swapper/120:41:291
2433025919,25sleep224372-21awk20:41:062
1431225918,23sleep114318-21diskstats23:35:571
97522584,19sleep10-21swapper/121:39:011
97522584,19sleep10-21swapper/121:39:011
952525816,24sleep29613-21sed23:21:122
33502586,43sleep00-21swapper/021:16:380
33502586,43sleep00-21swapper/021:16:380
3219425818,20sleep10-21swapper/121:06:091
2558025819,21sleep10-21swapper/100:11:071
2332825817,20sleep10-21swapper/122:21:091
230025822,20sleep20-21swapper/219:31:072
1666425810,33sleep016808-21unixbench_singl20:16:120
1492825814,20sleep00-21swapper/021:56:000
1492825814,20sleep00-21swapper/021:56:000
13182586,43sleep00-21swapper/019:26:530
1057125819,18sleep00-21swapper/021:41:060
57462575,11sleep20-21swapper/223:08:402
2849525715,27sleep028490-21apt-get00:20:520
2112825715,19sleep00-21swapper/022:15:430
1557825722,18sleep20-21swapper/221:56:142
1557825722,18sleep20-21swapper/221:56:142
1549025720,24sleep215529-21modprobe23:40:392
1123725712,21sleep20-21swapper/223:26:012
1003225717,23sleep110061-21memory19:56:061
755125614,25sleep17627-21systemctl21:31:091
47325617,17sleep00-21swapper/000:35:420
2888625613,28sleep128940-21sed20:56:031
2461625615,24sleep324611-21apt-get19:00:553
203225617,21sleep00-21swapper/022:56:260
1374825612,19sleep10-21swapper/121:51:091
1171625614,25sleep011711-21apt-get21:45:540
3126225512,25sleep031257-21apt-get19:20:550
3126225512,25sleep031257-21apt-get19:20:550
2208025512,29sleep022105-21sh00:01:010
190325514,25sleep01896-21apt-get19:30:560
109442556,37sleep00-21swapper/020:00:330
832725414,24sleep18384-21irqstats19:51:011
3219025414,19sleep20-21swapper/222:50:592
3200025413,24sleep132143-21modprobe22:50:421
2078225415,23sleep020801-21apt-get20:30:500
749725315,20sleep00-21swapper/023:15:410
574825313,23sleep05771-21irqstats21:26:030
460225317,17sleep20-21swapper/219:38:582
3103625315,22sleep10-21swapper/122:46:051
3103625315,22sleep10-21swapper/122:46:051
1006625314,25sleep210128-21/usr/sbin/munin23:21:242
1639425211,25sleep216390-21apt-get22:00:552
1639425211,25sleep216390-21apt-get22:00:552
618925112,21sleep16184-21apt-get23:10:531
603625112,21sleep16108-21sensors_temp21:26:131
2907825112,22sleep129084-21apt-get22:40:521
2365225114,20sleep10-21swapper/100:05:581
1878025116,19sleep00-21swapper/020:25:410
1616925115,21sleep016211-21ntpq23:41:050
1616925115,21sleep016211-21ntpq23:41:050
2999025012,14sleep10-21swapper/119:16:031
2812825012,22sleep128125-21apt-get19:10:541
2285725014,21sleep222853-21apt-get22:20:542
1658025016,19sleep116627-21basename22:01:021
1658025016,19sleep116627-21basename22:01:021
728924918,13sleep10-21swapper/119:46:121
54462497,15sleep00-21swapper/023:06:120
2902524912,22sleep028591-21/usr/sbin/munin20:56:060
285952498,16sleep20-21swapper/200:21:102
2745724813,20sleep227493-21ntp_states20:51:082
2100924816,17sleep00-21swapper/023:56:120
1948324811,22sleep219479-21apt-get22:10:522
2180224711,15sleep20-21swapper/200:00:422
562924515,15sleep10-21swapper/119:41:111
2137124310,16sleep20-21swapper/222:15:572
3012624211,16sleep10-21swapper/100:25:581
1910524210,17sleep20-21swapper/223:51:012
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional