You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-23 - 20:01

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7s.osadl.org (updated Wed Sep 23, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
142102223718,2199sleep08807-21kworker/u8:307:52:380
142102223718,2199sleep08807-21kworker/u8:307:52:380
5219219006,1882sleep34914-21kworker/3:112:32:083
5219219006,1882sleep34914-21kworker/3:112:32:083
7904246715,436sleep07913-21/usr/sbin/munin07:32:520
7904246715,436sleep07913-21/usr/sbin/munin07:32:520
32141237512,350sleep032227-21mailstats08:47:560
32141237512,350sleep032227-21mailstats08:47:560
28167235016,318sleep028205-21/usr/sbin/munin10:17:560
5689232340,268sleep35891-21latency_hist07:27:293
13841229830,255sleep313729-21latency_hist07:52:293
13841229830,255sleep313729-21latency_hist07:52:293
27786225810,232sleep327803-21fschecks_count10:17:473
22776224629,194sleep322914-21latency_hist10:02:283
24108222910,204sleep324203-21timerandwakeup11:48:023
24108222910,204sleep324203-21timerandwakeup11:48:023
12215222956,101sleep312266-21latency_hist07:47:283
26393222443,123sleep326490-21grep11:57:273
19512221822,177sleep119694-21latency_hist09:52:291
32032219714,121sleep332097-21awk07:07:553
18584218710,162sleep318739-21latency_hist11:32:283
29962218441,62sleep30-21swapper/312:07:453
1663521836,12sleep00-21swapper/011:23:000
1663521836,12sleep00-21swapper/011:23:000
6063218110,18sleep20-21swapper/207:27:452
6912217629,88sleep36929-21modprobe09:12:263
28925217235,74sleep30-21swapper/308:38:193
28925217235,74sleep30-21swapper/308:38:193
32606216859,54sleep30-21swapper/310:32:493
29719216830,123sleep329867-21sed12:07:293
29719216830,123sleep329867-21sed12:07:293
8788216618,134sleep38795-21wc09:17:433
2037021666,14sleep00-21swapper/011:37:380
18710216619,131sleep118655-21latency_hist08:07:281
7922216517,133sleep38076-21grep09:12:583
128321629,108sleep31363-21sed08:53:003
2150215963,40sleep30-21swapper/312:22:273
31237215831,100sleep231245-21modprobe07:07:262
31237215831,100sleep231245-21modprobe07:07:262
11645215721,113sleep111797-21unixbench_singl07:42:581
12646215038,59sleep30-21swapper/309:28:003
7702214521,52sleep30-21swapper/310:57:273
7702214521,52sleep30-21swapper/310:57:273
9258214312,116sleep39366-21latency_hist11:02:283
5232214115,110sleep35361-21timerwakeupswit10:48:023
20945213828,94sleep321006-21sendmail_mailqu08:12:563
27379213513,68sleep3231-21systemd-journal10:17:263
16273213129,42sleep30-21swapper/311:22:553
16273213129,42sleep30-21swapper/311:22:553
9769213050,17sleep30-21swapper/307:38:163
1365213016,93sleep01478-21grep08:53:010
1312213017,60sleep30-21swapper/307:12:483
6869212915,101sleep06955-21tr10:52:560
6869212915,101sleep06955-21tr10:52:560
12853212819,21sleep30-21swapper/311:12:503
12853212819,21sleep30-21swapper/311:12:503
3705212712,95sleep33854-21dpkg09:02:283
3705212712,95sleep33854-21dpkg09:02:283
19290212665,16sleep30-21swapper/308:07:513
22325212445,24sleep30-21swapper/308:17:483
1829212322,86sleep21844-21needreboot10:37:482
1829212322,86sleep21844-21needreboot10:37:482
3296212131,46sleep33373-21latency10:42:503
3296212131,46sleep33373-21latency10:42:503
29772211954,20sleep30-21swapper/310:22:553
2368821196,100sleep30-21swapper/308:22:453
20987211929,41sleep30-21swapper/311:37:573
26465211818,82sleep226570-21latency_hist08:32:282
13928211812,21sleep20-21swapper/209:32:562
576211637,63sleep3231-21systemd-journal12:17:263
26414211619,79sleep00-21swapper/008:32:260
14974211258,37sleep30-21swapper/309:37:373
976621116,14sleep30-21swapper/311:02:483
4682211112,76sleep14740-21/usr/sbin/munin07:22:551
9956211013,79sleep09988-21/usr/sbin/munin11:02:530
12259211015,79sleep012304-21grep09:27:530
31310210811,84sleep031392-21latency_hist07:07:280
31310210811,84sleep031392-21latency_hist07:07:280
350210625,61sleep1392-21wc10:32:551
1708321069,61sleep317113-21basename08:02:263
1442621058,79sleep314466-21grep07:53:183
1442621058,79sleep314466-21grep07:53:183
29926210414,74sleep130091-21sed10:22:571
15723210420,70sleep115758-21cpu11:22:411
15723210420,70sleep115758-21cpu11:22:411
31844210228,17sleep30-21swapper/310:29:423
17039210227,58sleep317122-21mailstats09:42:543
32043210117,65sleep132095-21latency12:13:041
22024210114,70sleep222063-21fschecks_count11:42:442
5714210012,72sleep05767-21/usr/sbin/munin12:32:450
2867529916,71sleep228716-21fschecks_time08:38:142
2867529916,71sleep228716-21fschecks_time08:38:142
147729941,14sleep30-21swapper/310:37:413
147729941,14sleep30-21swapper/310:37:413
1590829817,59sleep116000-21sort07:57:441
193552979,52sleep319408-21apt-config09:52:273
398529617,62sleep34132-21latency_hist07:22:293
3162829638,37sleep30-21swapper/308:47:433
3162829638,37sleep30-21swapper/308:47:433
2976929532,32sleep10-21swapper/107:02:291
2976929532,32sleep10-21swapper/107:02:291
611629428,49sleep30-21swapper/307:27:503
561729427,17sleep30-21swapper/312:32:443
339629415,63sleep13444-21/usr/sbin/munin10:42:521
339629415,63sleep13444-21/usr/sbin/munin10:42:521
280729413,61sleep22636-21/usr/sbin/munin07:17:472
2587029421,52sleep125917-21latency_hist10:12:281
2427229420,58sleep024296-21modprobe10:07:260
230329421,41sleep12295-21apt-get08:57:421
230329421,41sleep12295-21apt-get08:57:421
1219829413,18sleep30-21swapper/311:11:083
1138129414,64sleep211447-21sendmail_mailst07:42:552
3206029319,60sleep132074-21idleruntime-cro10:32:261
2751429313,65sleep127621-21timerwakeupswit11:58:101
1793029331,47sleep30-21swapper/308:02:583
3054829127,46sleep30-21swapper/308:42:553
406429011,25sleep30-21swapper/309:02:373
2331729021,21sleep10-21swapper/108:22:261
3016128913,60sleep330225-21/usr/sbin/munin07:02:473
3016128913,60sleep330225-21/usr/sbin/munin07:02:473
407828716,52sleep14152-21fschecks_time12:27:501
407828716,52sleep14152-21fschecks_time12:27:501
3250128614,22sleep20-21swapper/212:13:132
2015728619,50sleep220247-21latency_hist11:37:272
1524528611,21sleep215271-21irqstats09:37:482
273542858,51sleep30-21swapper/311:58:083
1939628424,28sleep00-21swapper/011:32:570
1648828415,48sleep015790-21/usr/sbin/munin07:57:550
1468328426,38sleep014687-21cron09:37:260
2191628320,45sleep121923-21/usr/sbin/munin09:57:541
695128222,30sleep20-21swapper/212:37:282
2320028121,12sleep30-21swapper/311:47:263
1557428117,20sleep10-21swapper/107:57:271
1557428117,20sleep10-21swapper/107:57:271
2192428016,48sleep221930-21open_files09:57:552
88002799,57sleep38835-21munin-run07:37:273
88002799,57sleep38835-21munin-run07:37:273
424927914,48sleep14252-21/usr/sbin/munin09:03:111
3138627925,25sleep00-21swapper/008:47:280
2696827915,30sleep20-21swapper/208:32:492
2696827915,30sleep20-21swapper/208:32:492
2121127911,16sleep30-21swapper/309:57:283
2047127820,28sleep20-21swapper/208:12:422
546527716,41sleep05485-21munin-run07:27:260
292827726,35sleep30-21swapper/312:22:553
292827726,35sleep30-21swapper/312:22:553
233042778,53sleep223450-21date11:47:292
233042778,53sleep223450-21date11:47:292
1712727714,30sleep00-21swapper/009:42:550
3055927614,30sleep00-21swapper/010:27:290
219827622,25sleep00-21swapper/012:22:280
1881827620,12sleep30-21swapper/311:32:403
720527510,48sleep00-21swapper/009:12:430
574627522,29sleep10-21swapper/109:07:471
2829827518,40sleep228378-21awk10:17:582
2667727518,12sleep30-21swapper/308:32:383
2667727518,12sleep30-21swapper/308:32:383
1989027517,38sleep119873-21apt-get09:52:451
1080727519,24sleep20-21swapper/211:07:272
810327418,38sleep28160-21/usr/sbin/munin10:57:462
2374027417,40sleep123886-21unixbench_singl10:02:591
1029627419,11sleep30-21swapper/309:22:313
762427319,36sleep17643-21idleruntime-cro10:57:251
762427319,36sleep17643-21idleruntime-cro10:57:251
321427316,35sleep03275-21/usr/sbin/munin07:17:580
2629527325,30sleep10-21swapper/111:57:251
2310727312,46sleep1374-21sshd11:46:151
1837027310,49sleep018422-21latency09:47:520
140082737,51sleep313402-21/usr/sbin/munin09:32:573
925627212,37sleep09319-21awk09:17:560
58627220,23sleep10-21swapper/107:12:291
2465427221,27sleep20-21swapper/210:07:482
2399927217,29sleep10-21swapper/108:22:511
245692719,39sleep30-21swapper/310:07:453
1507727115,13sleep30-21swapper/311:17:583
1507727115,13sleep30-21swapper/311:17:583
148027114,26sleep10-21swapper/108:53:021
1322127115,27sleep00-21swapper/007:47:580
1322127115,27sleep00-21swapper/007:47:580
426127022,22sleep00-21swapper/012:27:530
426127022,22sleep00-21swapper/012:27:530
2405827015,11sleep30-21swapper/310:04:423
2122327020,34sleep021353-21latency_hist09:57:290
1137527011,40sleep011418-21memory11:07:540
650226920,26sleep10-21swapper/107:28:001
162626918,33sleep2706-21/usr/sbin/munin08:53:032
1152726913,36sleep111585-21/usr/sbin/munin11:07:591
855526812,41sleep08675-21sed10:57:570
608326819,23sleep10-21swapper/112:32:541
3106926819,28sleep00-21swapper/010:27:530
949226718,30sleep19489-21apt-get11:02:381
583526714,37sleep25875-21http10:52:252
253682676,12sleep30-21swapper/308:27:493
633426612,32sleep20-21swapper/210:52:382
633426612,32sleep20-21swapper/210:52:382
2654526614,34sleep226564-21ntp_states10:12:532
1548526618,30sleep015455-21latency_hist11:22:270
1548526618,30sleep015455-21latency_hist11:22:270
1283026623,22sleep10-21swapper/107:47:511
1283026623,22sleep10-21swapper/107:47:511
934326516,30sleep29346-21apt-get07:37:382
781326520,24sleep20-21swapper/209:12:572
592926517,27sleep00-21swapper/009:07:500
506826517,25sleep10-21swapper/110:48:001
2971126522,23sleep20-21swapper/210:22:542
1417826515,32sleep214222-21/usr/sbin/munin11:17:412
1417826515,32sleep214222-21/usr/sbin/munin11:17:412
905826414,29sleep20-21swapper/209:17:502
3116826417,30sleep231213-21awk10:27:552
2174326422,22sleep20-21swapper/211:42:282
152026421,26sleep10-21swapper/112:17:561
1164726418,25sleep00-21swapper/009:27:290
100426416,34sleep21039-21sed07:12:352
791426313,22sleep10-21swapper/107:32:531
791426313,22sleep10-21swapper/107:32:531
2650226314,30sleep00-21swapper/011:57:280
2511326314,22sleep00-21swapper/010:07:590
2205826312,32sleep022102-21/usr/sbin/munin08:17:410
1949326316,23sleep10-21swapper/111:32:581
432926217,24sleep00-21swapper/010:47:270
432926217,24sleep00-21swapper/010:47:270
361526213,22sleep20-21swapper/209:02:272
361526213,22sleep20-21swapper/209:02:272
3232326218,23sleep00-21swapper/010:32:370
316012626,37sleep20-21swapper/208:47:382
316012626,37sleep20-21swapper/208:47:382
2900926216,24sleep20-21swapper/212:02:552
2900926216,24sleep20-21swapper/212:02:552
120312626,42sleep00-21swapper/007:45:440
386026116,19sleep00-21swapper/009:02:290
2397126117,26sleep00-21swapper/008:22:500
2170026113,33sleep121741-21latency_hist11:42:271
1468826122,22sleep10-21swapper/109:37:271
536326013,27sleep05482-21users10:48:030
3164026014,20sleep00-21swapper/007:07:440
2966726018,24sleep00-21swapper/012:07:250
2966726018,24sleep00-21swapper/012:07:250
2796326014,27sleep027990-21/usr/sbin/munin08:37:270
2796326014,27sleep027990-21/usr/sbin/munin08:37:270
2685326016,26sleep20-21swapper/211:57:422
253026012,22sleep20-21swapper/212:22:462
253026012,22sleep20-21swapper/212:22:462
1681826021,20sleep10-21swapper/109:42:481
1652026016,27sleep216540-21apt-get09:42:372
1500326015,24sleep00-21swapper/009:37:420
98725911,32sleep21077-21fschecks_time12:17:452
284852599,32sleep00-21swapper/008:37:350
284852599,32sleep00-21swapper/008:37:350
2647925915,25sleep00-21swapper/010:12:510
1892325915,28sleep218917-21apt-get08:07:422
1782825913,20sleep00-21swapper/008:02:550
1304825911,24sleep00-21swapper/011:12:550
1304825911,24sleep00-21swapper/011:12:550
902325813,21sleep10-21swapper/109:17:491
309325819,18sleep10-21swapper/112:22:571
309325819,18sleep10-21swapper/112:22:571
3006525817,25sleep130058-21apt-get08:42:411
2893725812,19sleep10-21swapper/112:02:541
2893725812,19sleep10-21swapper/112:02:541
210122586,12sleep00-21swapper/008:12:570
2024925817,19sleep10-21swapper/108:12:291
1727225813,27sleep117269-21apt-get11:27:411
1518425820,19sleep10-21swapper/109:37:471
947625714,29sleep19482-21/usr/sbin/munin07:38:101
854825715,20sleep10-21swapper/109:17:281
434425716,22sleep20-21swapper/209:03:282
2996125714,27sleep229957-21apt-get12:07:442
2661025712,18sleep10-21swapper/110:12:561
2200925718,18sleep00-21swapper/011:42:420
174525715,20sleep10-21swapper/110:37:471
174525715,20sleep10-21swapper/110:37:471
1375325716,23sleep213840-21gpgv07:52:282
1375325716,23sleep213840-21gpgv07:52:282
1235425716,26sleep212434-21latency_hist11:12:282
1211825716,25sleep112152-21irqstats09:27:501
428325616,25sleep24289-21diskstats07:22:442
3170425617,20sleep10-21swapper/108:47:451
3170425617,20sleep10-21swapper/108:47:451
3128525614,25sleep131431-21sed12:12:271
298692565,38sleep00-21swapper/012:07:300
2364225613,20sleep10-21swapper/111:47:521
2364225613,20sleep10-21swapper/111:47:521
232882566,38sleep20-21swapper/208:22:072
2173925618,18sleep10-21swapper/108:17:261
2023225621,20sleep00-21swapper/008:12:280
1985425616,15sleep20-21swapper/209:52:382
1585925617,16sleep20-21swapper/207:57:412
129912567,12sleep20-21swapper/211:12:532
129912567,12sleep20-21swapper/211:12:532
725325512,17sleep00-21swapper/007:32:270
365725512,32sleep1374-21sshd07:19:081
2848625513,25sleep127997-21apt-get08:37:361
2848625513,25sleep127997-21apt-get08:37:361
2803525518,23sleep128058-21munin-run08:37:281
2803525518,23sleep128058-21munin-run08:37:281
252442556,11sleep10-21swapper/108:27:471
1426125523,15sleep10-21swapper/111:17:441
1426125523,15sleep10-21swapper/111:17:441
1033325511,17sleep20-21swapper/209:22:422
938825418,18sleep00-21swapper/007:38:040
777925418,18sleep20-21swapper/207:32:492
777925418,18sleep20-21swapper/207:32:492
26812545,10sleep00-21swapper/010:41:420
26812545,10sleep00-21swapper/010:41:420
2423525414,18sleep00-21swapper/011:48:030
2423525414,18sleep00-21swapper/011:48:030
2360325416,22sleep223599-21apt-get08:22:392
1434525415,22sleep214356-21diskstats07:53:092
1434525415,22sleep214356-21diskstats07:53:092
109932549,29sleep111114-21sed09:22:581
696425319,19sleep27101-21date09:12:292
1775125310,24sleep20-21swapper/208:02:532
168125318,21sleep00-21swapper/007:12:570
1251925318,20sleep20-21swapper/209:27:592
447125217,18sleep20-21swapper/212:27:592
447125217,18sleep20-21swapper/212:27:592
344525212,23sleep23481-21ntp_states10:42:532
344525212,23sleep23481-21ntp_states10:42:532
2820625214,16sleep10-21swapper/110:17:571
251525212,23sleep02522-21diskstats12:22:440
251525212,23sleep02522-21diskstats12:22:440
231762526,33sleep222986-21/usr/sbin/munin10:02:482
3231225112,23sleep032384-21swap12:13:110
3224625114,20sleep20-21swapper/207:07:582
2169925114,19sleep00-21swapper/009:57:490
1809425112,23sleep218096-21apt-get09:47:422
139592519,15sleep10-21swapper/111:17:281
139592519,15sleep10-21swapper/111:17:281
3042625013,17sleep20-21swapper/208:42:502
2682125013,20sleep00-21swapper/011:57:360
1735125013,16sleep20-21swapper/211:27:462
802824914,15sleep10-21swapper/110:57:451
3201724911,15sleep10-21swapper/107:07:531
2540424914,19sleep20-21swapper/211:52:502
172572498,18sleep00-21swapper/011:27:400
123124820,15sleep20-21swapper/210:37:282
1699424711,21sleep217029-21systemd-run11:27:262
1699424711,21sleep217029-21systemd-run11:27:262
273624210,13sleep00-21swapper/008:57:530
273624210,13sleep00-21swapper/008:57:530
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional