You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-06 - 08:49

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Thu Aug 06, 2026 00:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
84502367011,3647sleep25245-21kworker/2:120:49:582
10711211477,1128sleep110742-21unin-run22:42:231
27088273721,701sleep227094-21df_inode23:32:412
777625459,501sleep17814-21ntpq20:47:471
2162623856,368sleep07700-1kworker/0:1H21:32:350
7557236017,325sleep27572-21apt22:32:222
166222296116,54sleep116767-21grep22:57:571
1565122799,256sleep115688-21sed22:57:241
1565122799,256sleep115688-21sed22:57:241
29189227725,222sleep129320-21grep23:37:531
3137822719,247sleep131419-21ntpq22:02:471
315712254100,136sleep17846-21cyclictest20:22:241
1595223724,193sleep11749-21unixbench_multi20:27:531
8845223612,209sleep07700-1kworker/0:1H00:17:350
7852223481,64sleep17853-21cpu22:32:381
7852223481,64sleep17853-21cpu22:32:381
7804223315,199sleep10-21swapper/119:07:251
13579222033,62sleep113469-21/usr/sbin/munin00:32:421
26487221619,180sleep326510-21idleruntime-cro23:32:223
13708221663,18sleep10-21swapper/121:07:411
26211221144,24sleep10-21swapper/120:02:591
3182220747,142sleep13330-21unixbench_singl20:32:541
29060220737,31sleep10-21swapper/120:12:481
24518220716,173sleep124593-21latency_hist21:42:251
25155220219,161sleep10-21swapper/121:42:491
29108219712,164sleep129159-21sendmail21:57:221
29108219712,164sleep129159-21sendmail21:57:221
1249219515,165sleep11311-21processes23:52:501
32370218783,60sleep10-21swapper/120:22:531
15272218219,110sleep10-21swapper/119:27:511
32513218111,154sleep131662-21/usr/sbin/munin23:47:541
9377216913,73sleep10-21swapper/119:12:251
9377216913,73sleep10-21swapper/119:12:251
21155216910,136sleep121181-21munin-run21:32:231
16845216314,134sleep116931-21timerandwakeup19:32:521
16845216314,134sleep116931-21timerandwakeup19:32:521
19635216216,133sleep119724-21latency19:42:451
545216020,126sleep1610-21grep22:07:511
545216020,126sleep1610-21grep22:07:511
19639215513,38sleep10-21swapper/123:07:541
12433215549,28sleep10-21swapper/119:17:541
14403215412,92sleep11-21systemd19:27:231
972521500,85sleep10-21swapper/120:52:551
282021498,67sleep10-21swapper/123:57:511
18389214940,72sleep118402-21fschecks_count21:22:431
18389214940,72sleep118402-21fschecks_count21:22:431
10578214414,76sleep10-21swapper/100:22:411
10578214414,76sleep10-21swapper/100:22:411
5462214213,87sleep15503-21grep00:07:351
1243521379,78sleep112471-21ntpq21:02:491
11693213722,82sleep10-21swapper/122:42:551
11188213612,53sleep111349-21awk20:57:521
11188213612,53sleep111349-21awk20:57:521
2968213213,70sleep12979-21ls22:17:241
27412213117,40sleep30-21swapper/320:07:453
16373213113,21sleep20-21swapper/222:57:542
9398213020,31sleep29618-21grep19:12:262
9398213020,31sleep29618-21grep19:12:262
5532212920,95sleep15561-21modprobe20:42:221
2403321298,42sleep10-21swapper/119:57:381
14334212726,83sleep314308-21apt-get22:52:503
14334212726,83sleep314308-21apt-get22:52:503
5404212655,20sleep10-21swapper/122:22:531
10632212512,98sleep210665-21irqstats00:22:422
10632212512,98sleep210665-21irqstats00:22:422
6300212339,12sleep10-21swapper/122:27:371
27136212210,38sleep10-21swapper/120:07:351
18710212214,94sleep218715-21modprobe23:07:222
16518212117,47sleep016457-21latency_hist21:17:250
22460211711,36sleep10-21swapper/119:52:381
22460211711,36sleep10-21swapper/119:52:381
21510211514,81sleep021595-21awk19:47:560
21051211418,75sleep10-21swapper/123:12:501
21051211418,75sleep10-21swapper/123:12:501
1381211418,79sleep21446-21awk20:27:502
2089821105,93sleep00-21swapper/021:29:400
10105211019,68sleep210128-21munin-run00:22:222
13789210614,76sleep013828-21apt-get22:52:230
10292210614,26sleep310385-21memory19:12:483
22530210520,41sleep10-21swapper/123:17:471
17442210434,49sleep10-21swapper/123:02:331
23767210313,74sleep323827-21fschecks_time23:22:413
22889210319,40sleep10-21swapper/121:37:201
17910210315,73sleep317998-21fschecks_time19:37:463
17910210315,73sleep317998-21fschecks_time19:37:463
12282210319,57sleep012429-21grep19:17:530
7049210118,47sleep10-21swapper/120:46:581
27503210123,31sleep10-21swapper/121:51:371
25422210014,71sleep325549-21timerwakeupswit21:42:553
23445210018,69sleep323522-21cut23:22:253
14827210017,67sleep314951-21latency_hist00:37:243
307629915,59sleep23225-21sed23:57:562
2760429914,70sleep327640-21latency_hist21:52:233
156032998,37sleep10-21swapper/121:12:501
2648529719,30sleep10-21swapper/123:32:201
2538629714,67sleep025537-21latency_hist20:02:250
52132967,15sleep10-21swapper/100:07:231
2940029636,41sleep10-21swapper/121:57:341
2466329623,27sleep10-21swapper/123:23:481
1789829621,32sleep10-21swapper/119:37:441
1789829621,32sleep10-21swapper/119:37:441
144729612,64sleep31516-21/usr/sbin/munin20:27:513
411629519,27sleep10-21swapper/122:17:571
301412957,73sleep10-21swapper/123:42:371
162112959,13sleep00-21swapper/019:32:300
162112959,13sleep00-21swapper/019:32:300
1769929419,27sleep10-21swapper/121:18:191
1769929419,27sleep10-21swapper/121:18:191
1260629420,31sleep10-21swapper/122:47:391
2922229313,60sleep228665-21/usr/sbin/munin20:12:532
1100329316,32sleep10-21swapper/119:13:551
1026729315,64sleep210362-21sed20:57:252
713129229,34sleep30-21swapper/320:47:253
346829211,67sleep33470-21sshd00:00:233
3067129213,61sleep230742-21sensors_temp23:42:502
1753629216,63sleep017588-21fschecks_count23:02:400
90592918,12sleep10-21swapper/100:17:451
61529033,28sleep30-21swapper/320:27:233
2980129015,30sleep10-21swapper/123:41:441
2446729018,56sleep324470-21cron21:42:223
1113029016,48sleep210960-21/usr/sbin/munin22:42:442
1632728926,42sleep216407-21fschecks_time19:32:412
1632728926,42sleep216407-21fschecks_time19:32:412
298128716,52sleep23131-21latency_hist22:17:252
72732868,31sleep10-21swapper/100:12:441
3243828615,24sleep20-21swapper/220:22:542
288632844,68sleep00-21swapper/021:54:100
288632844,68sleep00-21swapper/021:54:100
2031128417,15sleep00-21swapper/023:12:250
1351828312,47sleep20-21swapper/219:22:482
1325128317,42sleep313215-21latency_hist00:32:243
1215228323,32sleep00-21swapper/021:02:440
199162820,11sleep00-21swapper/019:42:510
130432826,33sleep10-21swapper/100:30:201
255542819,19sleep00-21swapper/021:42:560
2255828115,48sleep222665-21grep19:52:432
2255828115,48sleep222665-21grep19:52:432
212022819,23sleep17846-21cyclictest19:47:491
1843828115,50sleep218564-21timerwakeupswit19:37:572
1843828115,50sleep218564-21timerwakeupswit19:37:572
171892818,18sleep30-21swapper/323:02:233
605228025,28sleep00-21swapper/022:27:240
282928013,17sleep20-21swapper/220:32:472
2011928014,49sleep220259-21/usr/sbin/munin19:42:542
1955928014,48sleep019588-21awk21:27:220
1955928014,48sleep019588-21awk21:27:220
2995427913,50sleep029956-21cron20:17:230
2560127912,54sleep225624-21grep20:02:442
1252927916,38sleep212523-21apt-get22:47:362
1018727922,32sleep00-21swapper/020:57:230
330827810,54sleep03313-21cron23:58:220
3202227812,50sleep032054-21memory23:47:450
1090327815,46sleep310945-21ntp_offset20:57:463
1090327815,46sleep310945-21ntp_offset20:57:463
1536927720,27sleep20-21swapper/219:27:522
1508827711,50sleep015124-21/usr/sbin/munin19:27:460
86927622,32sleep00-21swapper/023:52:410
34072766,13sleep00-21swapper/020:33:210
288827620,36sleep32913-21munin-run22:17:223
1676027612,30sleep00-21swapper/021:17:410
1676027612,30sleep00-21swapper/021:17:410
271402759,52sleep027161-21cpu20:07:380
1830627524,31sleep20-21swapper/221:22:402
1830627524,31sleep20-21swapper/221:22:402
2563327418,35sleep20-21swapper/223:27:492
2473127416,37sleep224732-21/usr/sbin/munin21:42:342
2235427424,33sleep20-21swapper/221:33:212
2091927420,31sleep20-21swapper/219:47:422
1170627415,26sleep00-21swapper/000:27:240
1170627415,26sleep00-21swapper/000:27:240
772927323,30sleep37764-21munin-run19:07:233
989327217,34sleep00-21swapper/022:37:490
828627221,32sleep38306-21apt00:17:223
3165927218,27sleep30-21swapper/322:02:533
2245827220,36sleep022453-21apt-get19:52:360
2245827220,36sleep022453-21apt-get19:52:360
1568927218,24sleep20-21swapper/222:57:252
1568927218,24sleep20-21swapper/222:57:252
1332027217,41sleep213343-21modprobe21:07:222
803827122,27sleep20-21swapper/222:32:432
803827122,27sleep20-21swapper/222:32:432
74992710,8sleep00-21swapper/020:47:410
3271727114,26sleep00-21swapper/020:22:560
2591027116,34sleep026050-21unixbench_multi23:27:550
2135627125,29sleep321120-21apt-get21:32:253
2063127126,25sleep30-21swapper/323:12:403
2063127126,25sleep30-21swapper/323:12:403
831427013,44sleep08395-21readlink00:17:230
812527016,39sleep08208-21sed22:32:450
812527016,39sleep08208-21sed22:32:450
59772708,50sleep25978-21sshd22:26:582
3145127022,25sleep00-21swapper/023:47:240
2030027014,39sleep320309-21munin-run23:12:243
1192427014,42sleep211937-21basename00:27:342
446226919,31sleep34486-21modprobe22:22:223
415926912,41sleep24193-21irqstats00:02:482
778226820,30sleep07800-21munin-run19:07:240
494026818,28sleep30-21swapper/322:22:433
379726819,21sleep30-21swapper/320:37:243
20222688,46sleep00-21swapper/022:12:470
767826713,36sleep37739-21memory20:47:453
2825826721,24sleep20-21swapper/221:52:512
2825826721,24sleep20-21swapper/221:52:512
2499726719,24sleep20-21swapper/223:27:242
1409026724,24sleep20-21swapper/221:07:502
138326715,25sleep20-21swapper/222:12:252
138326715,25sleep20-21swapper/222:12:252
1095026717,24sleep00-21swapper/020:57:470
1095026717,24sleep00-21swapper/020:57:470
769226618,28sleep20-21swapper/200:12:532
32892657,13sleep20-21swapper/222:17:412
3229126522,18sleep30-21swapper/323:47:523
310572658,16sleep20-21swapper/222:02:402
2413326518,20sleep20-21swapper/219:57:432
1830726521,25sleep00-21swapper/019:37:550
1830726521,25sleep00-21swapper/019:37:550
950626417,21sleep30-21swapper/320:52:543
450226418,19sleep00-21swapper/020:37:570
238292648,12sleep00-21swapper/023:22:420
1087126418,24sleep20-21swapper/220:57:452
1087126418,24sleep20-21swapper/220:57:452
2759026316,26sleep327667-21wc23:32:543
2378326320,17sleep20-21swapper/221:37:542
1909826320,24sleep20-21swapper/223:07:422
1469626323,21sleep30-21swapper/319:27:373
99326210,32sleep21050-21irqstats23:52:432
57962627,32sleep00-21swapper/020:42:330
2982626222,22sleep30-21swapper/321:57:483
2918726220,27sleep329290-21latency_hist21:57:243
2918726220,27sleep329290-21latency_hist21:57:243
2266726214,28sleep30-21swapper/319:52:443
2266726214,28sleep30-21swapper/319:52:443
3074326110,13sleep30-21swapper/323:42:513
2627926112,33sleep226298-21apt-get21:47:342
2178326114,25sleep30-21swapper/321:33:023
1650826115,26sleep00-21swapper/022:57:560
1289826114,28sleep312907-21munin-run19:22:243
1011526117,20sleep00-21swapper/019:12:450
629926014,27sleep06293-21apt-get22:27:360
452926019,28sleep30-21swapper/300:02:573
357526014,32sleep03623-21ntpq22:17:470
2878026013,33sleep028838-21latency23:37:440
2721626020,21sleep20-21swapper/220:07:402
266712607,42sleep00-21swapper/020:05:040
1954826015,28sleep319587-21sort19:42:433
1525726020,22sleep20-21swapper/221:12:432
1267926014,29sleep312800-21sed21:02:553
2039225916,16sleep20-21swapper/221:27:522
1870025915,22sleep30-21swapper/321:22:493
1870025915,22sleep30-21swapper/321:22:493
138425913,31sleep31461-21grep23:52:523
1010625914,27sleep30-21swapper/322:37:533
428425813,24sleep00-21swapper/000:02:500
2942525814,27sleep029420-21apt-get21:57:370
2367225817,26sleep00-21swapper/021:37:510
2270625815,27sleep222747-21/usr/sbin/munin23:17:522
1406925822,20sleep00-21swapper/000:32:530
3095625718,19sleep00-21swapper/022:02:340
2422625712,30sleep324283-21if_eth019:57:443
2054925719,17sleep00-21swapper/023:12:370
2054925719,17sleep00-21swapper/023:12:370
1670725721,21sleep30-21swapper/319:32:503
1670725721,21sleep30-21swapper/319:32:503
1625125715,24sleep30-21swapper/321:14:583
91225612,26sleep0907-21apt-get20:27:360
882025616,22sleep38306-21apt-get00:17:343
3250325613,26sleep332521-21apt22:07:333
3250325613,26sleep332521-21apt22:07:333
273412565,17sleep00-21swapper/021:49:300
211402569,31sleep221222-21find23:12:512
211402569,31sleep221222-21find23:12:512
1364425616,19sleep30-21swapper/319:22:533
757725516,25sleep37606-21munin-run22:32:233
601125510,18sleep20-21swapper/220:42:562
546125514,24sleep05455-21apt-get00:07:340
427725513,20sleep20-21swapper/220:37:392
2989625513,26sleep00-21swapper/023:42:240
2142125516,23sleep321504-21mailstats19:47:553
1737525512,26sleep30-21swapper/321:17:563
1737525512,26sleep30-21swapper/321:17:563
1493225513,23sleep00-21swapper/021:12:240
1362125514,23sleep313618-21apt-get21:07:353
122202558,37sleep00-21swapper/022:47:060
1167225515,24sleep211681-21df_inode19:17:392
1084825513,25sleep310855-21ntp_states00:22:473
1084825513,25sleep310855-21ntp_states00:22:473
806225418,16sleep30-21swapper/319:07:393
806225418,16sleep30-21swapper/319:07:393
3159325413,22sleep20-21swapper/220:22:252
3027225414,24sleep330265-21apt-get20:17:383
1752925414,23sleep317534-21diskstats23:02:393
935825311,27sleep09374-21systemctl19:12:240
935825311,27sleep09374-21systemctl19:12:240
588425312,24sleep35913-21ntp_states00:07:463
1428025314,21sleep214395-21users00:32:552
2222925215,24sleep022330-21grep23:17:410
792925111,24sleep37938-21diskstats22:32:403
792925111,24sleep37938-21diskstats22:32:403
557825116,20sleep30-21swapper/320:42:243
3252425114,21sleep032523-21apt-get22:07:340
3252425114,21sleep032523-21apt-get22:07:340
1341925112,23sleep013422-21irqstats19:22:460
1046125113,22sleep210472-21ntp_states19:12:512
986125012,16sleep20-21swapper/222:37:472
695625018,17sleep20-21swapper/219:02:502
317562507,19sleep20-21swapper/223:47:392
2989425017,19sleep20-21swapper/221:57:512
1717525011,22sleep217212-21ntp_states21:17:512
1717525011,22sleep217212-21ntp_states21:17:512
2914424912,22sleep329150-21ntp_states20:12:503
594724813,23sleep26024-21tr00:07:492
2974024810,24sleep329742-21sshd23:40:583
1202024813,21sleep012026-21df_inode00:27:410
1098224711,19sleep310979-21apt-get22:42:373
2562524610,21sleep325672-21df20:02:453
1552724612,21sleep0364-21snmpd22:56:430
1552724612,21sleep0364-21snmpd22:56:430
125302445,13sleep30-21swapper/322:47:373
310262408,13sleep00-21swapper/020:17:550
295292288,12sleep20-21swapper/223:38:142
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional