You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-01 - 05:41

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Fri May 01, 2026 00:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
287522514713,5122sleep323918-21kworker/3:022:40:083
16736212635,1247sleep011284-1kworker/0:2H22:00:520
16691269812,673sleep36091-21kworker/3:123:44:083
16691269812,673sleep36091-21kworker/3:123:44:083
23493256022,403sleep00-21swapper/000:08:080
31325397,517sleep0363-21ntpq22:50:550
31325397,517sleep0363-21ntpq22:50:550
3104023719,345sleep331114-21find00:30:523
244702358119,214sleep00-21swapper/000:10:460
84112283118,21sleep00-21swapper/023:16:030
2959822779,194sleep029606-21sh19:15:490
2797122429,219sleep01-21systemd20:55:230
3677223810,215sleep03690-21cpu21:20:380
23689223317,200sleep123707-21fschecks_count20:40:421
13644222414,193sleep113677-21ntp_kernel_pll_21:51:001
13644222414,193sleep113677-21ntp_kernel_pll_21:51:001
22419221968,80sleep022426-21needreboot20:35:440
22419221968,80sleep022426-21needreboot20:35:440
1393022176,198sleep00-21swapper/020:09:560
32717220586,43sleep0315-21wc19:25:500
21383220440,143sleep00-21swapper/022:15:420
19971218916,154sleep025408-21cyclictest22:10:450
30692218128,100sleep025408-21cyclictest19:20:370
3142121806,13sleep00-21swapper/000:31:500
31488217915,147sleep031509-21fschecks_count21:05:400
16198217410,151sleep016288-21apt-config22:00:240
14562217414,141sleep014590-21latency_hist21:55:230
14562217414,141sleep014590-21latency_hist21:55:230
24634217023,129sleep024759-21timerandwakeup19:00:510
787821668,142sleep07935-21grep19:50:420
2951521646,14sleep00-21swapper/022:40:500
2951521646,14sleep00-21swapper/022:40:500
27959216427,59sleep028080-21timerandwakeup00:20:550
25801216216,24sleep00-21swapper/019:05:430
25801216216,24sleep00-21swapper/019:05:430
23364216117,129sleep323421-21latency_hist20:40:233
23364216117,129sleep323421-21latency_hist20:40:233
182712161111,29sleep00-21swapper/020:20:540
17452215912,24sleep00-21swapper/023:45:510
17452215912,24sleep00-21swapper/023:45:510
5831215753,28sleep05856-21munin-run19:45:250
614721557,136sleep06166-21apt-get23:10:330
614721557,136sleep06166-21apt-get23:10:330
23403215310,105sleep023497-21switchtime22:20:510
23403215310,105sleep023497-21switchtime22:20:510
30203215011,89sleep00-21swapper/000:30:240
19190215013,118sleep00-21swapper/023:51:040
16789214911,17sleep00-21swapper/023:45:220
16789214911,17sleep00-21swapper/023:45:220
289062146112,16sleep00-21swapper/019:15:240
2145521455,46sleep00-21swapper/020:31:440
2145521455,46sleep00-21swapper/020:31:440
10164214310,115sleep010179-21kernelversion21:40:450
775921415,11sleep00-21swapper/021:31:570
1630021419,98sleep016334-21ps20:15:490
320421405,11sleep00-21swapper/021:18:040
2700421405,62sleep00-21swapper/000:19:420
2622321395,10sleep00-21swapper/020:47:520
1652213916,110sleep01709-21cut22:55:520
1652213916,110sleep01709-21cut22:55:520
10629213871,54sleep010668-21http20:00:230
1026921365,27sleep00-21swapper/023:21:360
89521349,86sleep0936-21ntpq21:10:490
18697212820,82sleep318844-21latency_hist20:25:253
17861212319,85sleep117862-21cron22:05:211
6344212286,22sleep06376-21fschecks_count19:45:570
4985212218,88sleep25039-21latency_hist21:25:252
2748921226,103sleep00-21swapper/022:35:330
27231212218,87sleep327270-21munin-node19:10:273
1893621228,70sleep00-21swapper/020:25:390
2936021215,45sleep00-21swapper/020:57:530
25880212119,85sleep225975-21perl00:15:412
25589212020,76sleep20-21swapper/200:15:242
1240521208,72sleep00-21swapper/023:30:400
220121199,94sleep22208-21sh21:15:402
3083211813,23sleep00-21swapper/023:00:440
23425211423,77sleep00-21swapper/020:40:250
30842211215,79sleep130339-21apt-get22:45:351
5101211023,64sleep15224-21timerwakeupswit19:40:541
29605211010,84sleep129749-21latency_hist21:00:241
27807210918,74sleep127822-21/usr/sbin/munin00:20:521
26575210910,79sleep00-21swapper/022:30:520
20027210721,66sleep220079-21latency_hist23:55:262
15921210721,67sleep315933-21fschecks_count20:15:403
1238221078,88sleep013441-21snmpd20:04:550
833721050,8sleep30-21swapper/323:16:013
1398221059,19sleep00-21swapper/023:35:370
8800210363,25sleep08808-21uname21:35:490
30726210270,20sleep030339-21apt-get22:45:260
2983621027,19sleep029838-21apt-get21:00:360
24381210219,69sleep324382-21cpu22:25:383
24381210219,69sleep324382-21cpu22:25:383
1153821028,17sleep00-21swapper/021:45:420
1153821028,17sleep00-21swapper/021:45:420
24419210018,64sleep324468-21irqstats00:10:453
132721007,15sleep00-21swapper/019:30:350
12160210021,52sleep312215-21latency_hist23:30:233
1845429815,68sleep218456-21sshd20:22:582
1221929825,29sleep00-21swapper/023:30:240
1489329727,49sleep214957-21sort21:55:412
146712977,15sleep00-21swapper/020:10:490
95829619,55sleep3999-21proc_pri21:10:523
258729524,31sleep32619-21modprobe19:35:223
2409929515,57sleep324180-21/usr/sbin/munin20:40:523
2019229417,57sleep320227-21idleruntime-cro20:30:223
184222936,11sleep00-21swapper/022:05:450
1541429312,64sleep115521-21timerandwakeup21:55:531
1114929332,37sleep311146-21apt-get20:00:343
445229229,38sleep20-21swapper/219:40:382
38692925,11sleep00-21swapper/019:36:360
57492915,12sleep00-21swapper/023:08:400
2107429127,29sleep30-21swapper/322:15:253
612129016,58sleep16133-21vmstat21:25:561
316529010,18sleep30-21swapper/323:00:483
3040228916,53sleep230513-21apt-key22:45:242
3040228916,53sleep230513-21apt-key22:45:242
2553228925,46sleep225575-21ntpq20:45:482
61922885,11sleep00-21swapper/021:26:400
2528928821,32sleep30-21swapper/319:05:223
1462328819,48sleep314229-21/usr/sbin/munin20:10:483
465828711,61sleep34708-21/usr/sbin/munin19:40:433
219702875,11sleep00-21swapper/000:03:270
254382866,12sleep00-21swapper/022:27:410
254382866,12sleep00-21swapper/022:27:410
2412028628,30sleep20-21swapper/219:00:412
202172867,64sleep00-21swapper/023:55:360
202172867,64sleep00-21swapper/023:55:360
152128611,61sleep21522-21/usr/sbin/munin22:55:382
152128611,61sleep21522-21/usr/sbin/munin22:55:382
802928523,45sleep28031-21cron21:35:222
3104428417,50sleep231050-21fschecks_count22:46:412
1277328421,42sleep212907-21latency_hist21:50:242
1277328421,42sleep212907-21latency_hist21:50:242
2738828315,17sleep10-21swapper/119:10:291
2753928214,51sleep227551-21apt-get19:10:342
904628113,53sleep19116-21systemd-run19:55:231
3202228123,29sleep30-21swapper/319:25:253
2761028119,37sleep127701-21fschecks_time22:35:421
2624928125,27sleep20-21swapper/219:05:542
2624928125,27sleep20-21swapper/219:05:542
1248228020,42sleep112630-21latency_hist20:05:251
351327919,41sleep10-21swapper/119:35:531
2678827812,52sleep226793-21df20:50:402
1288527823,30sleep10-21swapper/123:30:581
2639927722,34sleep126483-21wc00:15:521
2182027720,28sleep10-21swapper/120:35:251
2182027720,28sleep10-21swapper/120:35:251
139212770,57sleep30-21swapper/321:51:063
139212770,57sleep30-21swapper/321:51:063
685327623,32sleep16918-21uniq21:30:411
956927522,28sleep10-21swapper/119:55:431
749927532,26sleep10-21swapper/119:50:221
3265627516,44sleep132693-21ntp_states19:25:471
104592747,13sleep10-21swapper/123:24:031
2175827312,15sleep30-21swapper/300:00:443
178652739,49sleep217910-21systemd-run22:05:222
130712739,44sleep20-21swapper/220:05:462
285602728,42sleep30-21swapper/320:55:463
266827223,29sleep10-21swapper/119:35:241
2409127222,32sleep324109-21awk22:25:223
2409127222,32sleep324109-21awk22:25:223
2019227222,23sleep20-21swapper/223:55:342
2019227222,23sleep20-21swapper/223:55:342
1946327225,24sleep30-21swapper/322:10:243
246427116,27sleep10-21swapper/122:56:211
246427116,27sleep10-21swapper/122:56:211
2196227114,23sleep30-21swapper/322:15:533
1205827120,19sleep30-21swapper/321:45:533
1205827120,19sleep30-21swapper/321:45:533
759327018,27sleep30-21swapper/323:15:253
759327018,27sleep30-21swapper/323:15:253
3223827015,32sleep20-21swapper/219:25:362
825426916,21sleep20-21swapper/223:16:012
200952696,51sleep10-21swapper/120:29:071
3138626816,34sleep331421-21grep22:46:503
299226821,33sleep23060-21grep19:35:422
2750526813,38sleep227565-21/usr/sbin/munin00:20:452
946026723,27sleep30-21swapper/323:20:503
2693826716,34sleep127043-21sed19:10:251
2693826716,34sleep127043-21sed19:10:251
1434026717,19sleep20-21swapper/220:10:432
1165426716,29sleep20-21swapper/220:01:182
3169726620,29sleep131724-21munin-run00:35:221
2676226619,23sleep10-21swapper/120:50:391
2639826623,22sleep10-21swapper/122:30:491
2630726621,24sleep20-21swapper/222:30:462
2397126613,21sleep20-21swapper/220:40:472
2034426619,21sleep10-21swapper/122:10:551
194462666,42sleep20-21swapper/222:10:232
123352665,42sleep10-21swapper/121:45:551
123352665,42sleep10-21swapper/121:45:551
997926517,25sleep20-21swapper/223:21:012
3040226513,39sleep21-21systemd19:20:232
2727626522,20sleep30-21swapper/322:35:243
1765826512,40sleep217752-21cat23:45:572
1765826512,40sleep217752-21cat23:45:572
1629426516,30sleep116435-21/usr/sbin/munin23:41:031
1629426516,30sleep116435-21/usr/sbin/munin23:41:031
1452426523,20sleep10-21swapper/120:10:461
1071126521,20sleep20-21swapper/220:00:252
57726427,16sleep20-21swapper/221:10:432
3051626414,19sleep30-21swapper/322:45:253
2587826415,22sleep10-21swapper/120:45:551
2259526417,32sleep222725-21cat22:20:252
1891526413,32sleep119009-21memory23:50:561
1249326417,33sleep312512-21fschecks_count23:30:483
2465026320,21sleep20-21swapper/222:25:462
2465026320,21sleep20-21swapper/222:25:462
1861126313,27sleep10-21swapper/122:05:501
1445226320,31sleep214460-21awk23:35:592
1010526325,20sleep30-21swapper/321:40:443
3122326214,19sleep20-21swapper/219:20:522
3036026221,27sleep130401-21find22:45:231
3036026221,27sleep130401-21find22:45:231
302126221,22sleep10-21swapper/121:15:551
294122627,14sleep20-21swapper/220:58:322
24852627,18sleep30-21swapper/322:56:323
24852627,18sleep30-21swapper/322:56:323
1135026215,26sleep10-21swapper/123:26:011
680526119,25sleep36872-21sshd19:46:083
504326122,16sleep30-21swapper/321:25:263
3190726120,19sleep20-21swapper/221:05:512
3005026115,19sleep10-21swapper/121:00:451
2478326115,18sleep10-21swapper/122:25:491
2478326115,18sleep10-21swapper/122:25:491
1484026121,22sleep314893-21users23:36:043
2698126017,30sleep327054-21sed20:50:453
1883126015,26sleep30-21swapper/323:50:543
1838926018,21sleep10-21swapper/123:50:241
1838926018,21sleep10-21swapper/123:50:241
181226014,26sleep11847-21munin-run21:15:221
624625914,20sleep10-21swapper/123:10:431
624625914,20sleep10-21swapper/123:10:431
343825922,15sleep30-21swapper/321:20:253
2914425918,18sleep10-21swapper/119:15:381
2056625918,24sleep20-21swapper/220:30:412
2056625918,24sleep20-21swapper/220:30:412
2037525914,21sleep30-21swapper/323:56:113
2037525914,21sleep30-21swapper/323:56:113
1793025918,22sleep10-21swapper/120:20:511
783125815,24sleep10-21swapper/123:15:391
625625817,24sleep26259-21diskstats23:10:442
625625817,24sleep26259-21diskstats23:10:442
475325824,17sleep10-21swapper/123:05:501
30025817,26sleep3309-21/usr/sbin/munin22:50:543
30025817,26sleep3309-21/usr/sbin/munin22:50:543
2557725810,33sleep325614-21ntp_states20:45:493
2380125821,22sleep223986-21latency_hist00:10:242
2174025815,23sleep30-21swapper/320:35:233
2174025815,23sleep30-21swapper/320:35:233
1684025821,15sleep20-21swapper/223:45:242
1684025821,15sleep20-21swapper/223:45:242
1623525814,22sleep10-21swapper/120:15:461
1294525817,20sleep20-21swapper/223:30:592
1045325812,31sleep310454-21sshd19:58:083
989625715,19sleep10-21swapper/121:40:381
462125715,26sleep24618-21apt-get23:05:352
3199125714,23sleep30-21swapper/321:05:523
298762578,16sleep20-21swapper/200:26:512
2799925717,20sleep20-21swapper/222:35:512
2578825718,23sleep325785-21apt-get00:15:333
2484425714,23sleep10-21swapper/100:10:551
2066025715,24sleep120193-21/usr/sbin/munin23:56:221
2066025715,24sleep120193-21/usr/sbin/munin23:56:221
179025723,19sleep20-21swapper/219:30:492
1398325717,24sleep113978-21apt-get23:35:381
834425615,24sleep38340-21apt-get21:35:353
803425616,23sleep38106-21systemd-run21:35:233
58225621,18sleep20-21swapper/222:51:012
58225621,18sleep20-21swapper/222:51:012
3045225616,18sleep30-21swapper/319:20:243
2242725618,19sleep10-21swapper/120:35:451
2242725618,19sleep10-21swapper/120:35:451
221002567,17sleep20-21swapper/200:05:072
168702567,12sleep20-21swapper/220:17:292
1085525616,23sleep210878-21cpu23:25:382
449925512,28sleep14537-21latency_hist23:05:261
2292525524,17sleep20-21swapper/222:20:412
2292525524,17sleep20-21swapper/222:20:412
2212125514,23sleep10-21swapper/100:05:221
805525421,20sleep30-21swapper/319:50:453
486925414,17sleep30-21swapper/323:05:523
36625412,24sleep1398-21ntp_states22:50:561
36625412,24sleep1398-21ntp_states22:50:561
3256925417,22sleep332608-21memory19:25:453
2897225413,17sleep10-21swapper/100:25:411
2860425420,21sleep30-21swapper/300:25:233
2414325418,22sleep10-21swapper/122:25:241
2414325418,22sleep10-21swapper/122:25:241
1933625417,20sleep20-21swapper/220:25:492
1319825416,22sleep212717-21apt-get21:50:322
1319825416,22sleep212717-21apt-get21:50:322
1119425416,23sleep211203-21munin-run21:45:242
841425313,24sleep28421-21df_inode21:35:382
530425313,21sleep25308-21diskstats21:25:412
367325313,23sleep13670-21apt-get21:20:351
315825313,23sleep23164-21diskstats23:00:472
306125310,16sleep30-21swapper/319:35:433
3011425316,23sleep330208-21memory21:00:473
1611325315,19sleep30-21swapper/322:00:233
3110125215,21sleep20-21swapper/221:05:222
2104025216,16sleep20-21swapper/222:15:242
625725113,22sleep26225-21apt-get19:45:512
3095225121,16sleep20-21swapper/200:30:512
1708025114,22sleep317076-21apt-get23:45:373
1708025114,22sleep317076-21apt-get23:45:373
394325017,18sleep30-21swapper/321:20:453
292802509,14sleep30-21swapper/322:40:453
2561825019,17sleep30-21swapper/319:05:383
2561825019,17sleep30-21swapper/319:05:383
2329425017,18sleep10-21swapper/100:05:591
2240525014,18sleep30-21swapper/300:05:323
1277325011,14sleep30-21swapper/320:05:393
3067424912,21sleep130686-21apt-get19:20:341
2887124913,20sleep328889-21apt-get00:25:353
2411324916,17sleep10-21swapper/119:00:401
2212924812,22sleep222135-21df_inode20:35:382
2212924812,22sleep222135-21df_inode20:35:382
2135324811,15sleep10-21swapper/122:15:391
1690224815,16sleep30-21swapper/322:00:593
1270424811,23sleep112698-21apt-get20:05:371
3151124612,19sleep131609-21perl21:05:411
911824516,15sleep20-21swapper/219:55:242
83242398,15sleep10-21swapper/121:35:331
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional