You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-29 - 23:45

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot7s.osadl.org (updated Sun Mar 29, 2026 12:43:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
11200238716,352sleep011240-21/usr/sbin/munin09:32:390
1868023117,289sleep218721-21ntpq08:12:552
4189224592,139sleep24203-21kernelversion10:52:402
28372223223,191sleep128428-21tune2fs07:02:371
17028222364,84sleep217173-21latency_hist09:52:192
9906221717,91sleep29947-21grep09:27:452
27844221688,25sleep20-21swapper/208:42:262
1630221612,188sleep21764-21unixbench_multi07:17:482
24189221413,177sleep224345-21latency_hist11:57:202
1856222008,22sleep20-21swapper/211:37:422
14004219016,24sleep00-21swapper/011:22:460
9357217610,62sleep28816-21/usr/sbin/munin11:07:462
7988217616,57sleep20-21swapper/211:02:492
430021758,157sleep013441-21snmpd12:32:590
14870217130,124sleep215025-21latency_hist11:27:202
13877216929,127sleep213940-21latency_hist09:42:182
813321667,17sleep00-21swapper/009:22:380
5872216426,119sleep229661-21cyclictest07:32:422
2552216054,80sleep22693-21grep12:27:482
30097215937,99sleep20-21swapper/208:47:482
21625215735,27sleep221631-21munin-run10:07:172
29899215261,79sleep229926-21kernelversion12:13:032
1096721529,59sleep20-21swapper/209:32:292
21279214817,109sleep221276-21munin-run08:22:202
26725214685,39sleep20-21swapper/208:37:422
21047213951,26sleep20-21swapper/211:47:182
715213653,44sleep2852-21latency_hist07:17:192
6619213579,16sleep20-21swapper/209:17:392
1707821319,98sleep20-21swapper/211:32:452
4518213012,93sleep20-21swapper/209:12:182
7898212810,105sleep27903-21cpu09:22:322
22878212825,25sleep20-21swapper/208:26:522
31335212516,85sleep30-21swapper/312:17:363
31172212328,77sleep131215-21basename10:37:161
24492212217,88sleep00-21swapper/011:57:340
2134212234,66sleep20-21swapper/210:47:192
31061212032,71sleep20-21swapper/208:52:362
15581211719,25sleep20-21swapper/208:02:502
793211618,78sleep332764-21apt-get10:42:273
565211632,72sleep2634-21find08:57:452
3091211621,79sleep33133-21munin-run12:32:173
28854211435,56sleep328932-21switchtime07:02:473
19519211315,24sleep30-21swapper/311:42:193
8845211229,18sleep28929-21perl07:42:392
2923721128,11sleep00-21swapper/007:03:210
16325211230,18sleep216352-21munin-run11:32:162
12050211226,16sleep20-21swapper/211:17:352
3626211116,80sleep33636-21modprobe10:52:163
25676211117,30sleep225711-21modprobe12:02:162
1689921117,65sleep216946-21ntpq08:07:422
2751121107,60sleep20-21swapper/212:07:172
1084210912,68sleep00-21swapper/012:22:540
31181210823,19sleep20-21swapper/207:12:192
2254021067,27sleep20-21swapper/210:07:502
17296210613,25sleep00-21swapper/009:52:340
15143210614,72sleep115149-21df_inode11:27:351
24674210512,79sleep324705-21latency11:57:373
24461210513,28sleep20-21swapper/208:32:172
1641210514,76sleep01753-21unixbench_singl10:43:330
11562210516,69sleep111635-21/usr/sbin/munin09:32:481
848321049,18sleep20-21swapper/207:42:182
30448210411,77sleep030541-21sed07:07:480
14365210420,13sleep20-21swapper/209:42:392
6410210312,77sleep36532-21sed10:57:493
27580210120,61sleep027540-21latency_hist12:07:190
13480210124,14sleep20-21swapper/207:57:372
1916621008,18sleep20-21swapper/209:57:402
1861210016,69sleep31919-21/usr/sbin/munin09:02:383
603229913,62sleep06067-21munin-run09:17:170
2240729918,62sleep022556-21grep08:22:540
611229810,39sleep26219-21latency_hist09:17:192
312682989,73sleep231331-21grep12:17:352
1592729810,67sleep20-21swapper/209:47:392
2617429721,59sleep125692-21apt-get12:02:281
884129615,58sleep18894-21ls11:07:321
3244129614,59sleep131798-21/usr/sbin/munin07:12:501
2847429613,69sleep028601-21cat12:07:480
2984429515,65sleep329862-21apt07:07:273
24962959,19sleep22493-21apt-get07:22:282
1550229517,62sleep115579-21cat08:02:491
1257029514,59sleep112619-21/usr/sbin/munin09:37:331
1242929514,34sleep212552-21timerandwakeup07:52:482
3157929420,27sleep10-21swapper/110:37:371
265752949,16sleep20-21swapper/210:22:192
484629212,64sleep34853-21/usr/sbin/munin09:12:363
226329219,56sleep12351-21latency_hist07:22:191
328729111,22sleep33293-21df_inode09:07:353
29112918,71sleep22943-21munin-run09:07:182
2836129113,24sleep20-21swapper/210:27:322
236772918,67sleep20-21swapper/210:12:302
1047929022,40sleep310571-21fschecks_time11:12:353
3233228710,63sleep032334-21sshd12:20:520
2919028722,35sleep30-21swapper/312:12:193
2819428743,23sleep20-21swapper/212:07:422
37612867,11sleep20-21swapper/212:32:412
3402866,13sleep20-21swapper/212:22:352
2210228616,52sleep021620-21apt-get10:07:280
850428516,51sleep18612-21grep09:22:471
2931228520,40sleep029488-21apt-config12:12:200
2748928524,27sleep10-21swapper/110:22:471
1955428523,39sleep019574-21idleruntime-cro08:17:170
896328337,24sleep30-21swapper/309:24:513
3031328319,41sleep330453-21unixbench_multi12:13:123
1205328317,30sleep10-21swapper/107:52:391
1173128348,16sleep20-21swapper/211:17:192
631128211,52sleep20-21swapper/210:57:482
259228218,45sleep32475-21/usr/sbin/munin07:22:373
531028147,16sleep20-21swapper/207:32:172
531028147,16sleep20-21swapper/207:32:172
311352816,11sleep20-21swapper/210:36:492
2711528121,37sleep327127-21vmstat12:02:583
268112817,11sleep20-21swapper/210:22:322
23722817,59sleep20-21swapper/210:47:312
221128118,37sleep32226-21latency_hist07:22:173
1533928110,58sleep315341-21sshd09:46:513
2344228021,25sleep30-21swapper/310:12:193
1593728011,55sleep315938-21sshd08:03:523
1119028010,9sleep20-21swapper/207:47:562
1053728022,39sleep010568-21/usr/sbin/munin07:47:370
570627922,37sleep15196-21apt-get10:57:271
1421627919,37sleep014307-21fschecks_time09:42:370
311282789,58sleep00-21swapper/010:36:430
224092775,11sleep20-21swapper/211:50:332
1945527713,50sleep019582-21cat09:57:470
1078027721,37sleep00-21swapper/011:12:400
765927625,27sleep30-21swapper/311:02:423
325942765,11sleep20-21swapper/210:40:162
2651927619,39sleep326549-21idleruntime-cro10:22:163
30982750,6sleep00-21swapper/010:47:480
2812027522,29sleep00-21swapper/010:27:180
2452927522,29sleep10-21swapper/108:32:201
130072756,16sleep30-21swapper/309:37:433
2184727418,26sleep10-21swapper/108:22:431
1788627411,19sleep30-21swapper/308:12:193
438927316,40sleep04432-21cut10:52:460
1606627312,25sleep10-21swapper/109:47:421
1274427318,27sleep00-21swapper/011:17:490
2525227215,31sleep00-21swapper/008:32:480
1915427118,25sleep10-21swapper/109:57:391
1015827117,29sleep30-21swapper/311:12:193
994927021,23sleep30-21swapper/309:27:463
3105727024,29sleep131060-21/usr/sbin/munin08:52:351
258072705,54sleep20-21swapper/208:34:432
223192705,53sleep00-21swapper/011:49:220
162832709,48sleep016229-21latency_hist08:07:200
2641226913,27sleep00-21swapper/008:37:360
2034726915,30sleep10-21swapper/110:02:321
1063826914,39sleep110655-21irqstats11:12:371
3139626820,26sleep00-21swapper/007:12:210
313526819,28sleep00-21swapper/012:32:180
2843126831,21sleep20-21swapper/207:02:382
145626817,34sleep01548-21mailstats07:17:460
1264626826,22sleep30-21swapper/311:17:483
568926719,30sleep35783-21cut07:32:393
534226723,23sleep10-21swapper/110:57:191
348626717,24sleep00-21swapper/009:07:400
3145226724,19sleep10-21swapper/112:17:381
2794826718,27sleep10-21swapper/108:42:341
212602678,15sleep30-21swapper/311:47:283
1787526717,33sleep117906-21basename11:37:161
517826620,26sleep35252-21apt-config10:57:173
300026616,35sleep13146-21ls09:07:211
279572668,18sleep30-21swapper/310:25:523
2260726612,29sleep10-21swapper/110:07:531
1839826614,35sleep118500-21fschecks_time08:12:501
1255526618,26sleep30-21swapper/307:52:493
951926512,24sleep00-21swapper/011:07:480
765726517,20sleep10-21swapper/109:22:191
3255226511,39sleep132598-21grep08:57:321
2608826520,19sleep30-21swapper/308:37:193
191562657,46sleep00-21swapper/011:38:280
1795426519,21sleep30-21swapper/311:37:183
714926414,34sleep37168-21apt-get07:37:273
70626420,19sleep00-21swapper/008:57:470
2005926412,29sleep00-21swapper/008:17:370
1936026420,21sleep30-21swapper/309:57:463
466426321,24sleep10-21swapper/107:27:481
466426321,24sleep10-21swapper/107:27:481
43226315,19sleep30-21swapper/310:42:193
2991526318,28sleep329911-21apt-get10:32:303
1716826319,22sleep00-21swapper/011:32:470
1177026317,28sleep311892-21users09:32:503
893026219,22sleep30-21swapper/307:42:403
832026218,26sleep10-21swapper/107:40:521
2644826212,36sleep126534-21fw_packets08:37:371
2538526215,21sleep00-21swapper/010:17:370
2487526213,27sleep10-21swapper/111:57:431
209426223,21sleep10-21swapper/112:27:391
1827926216,29sleep318274-21apt-get08:12:323
81826112,33sleep1815-21apt-get10:42:291
3126726116,22sleep30-21swapper/307:12:203
3000226119,19sleep10-21swapper/108:47:471
298412617,13sleep10-21swapper/112:13:011
2835926116,27sleep028353-21apt-get10:27:300
274226117,27sleep12788-21/usr/sbin/munin10:47:421
2181326114,29sleep021974-21sed10:07:190
1974126115,32sleep119789-21awk11:42:331
1846726112,29sleep118561-21/usr/sbin/munin11:37:401
1778026120,23sleep30-21swapper/309:52:453
1504426122,19sleep30-21swapper/311:27:293
1399726114,23sleep10-21swapper/107:57:481
1218826117,29sleep012236-21sed07:52:420
494626019,21sleep10-21swapper/109:12:381
2044326016,21sleep10-21swapper/108:17:471
1999926014,18sleep30-21swapper/311:42:393
1767226013,30sleep117695-21/usr/sbin/munin09:52:431
1613326013,29sleep016171-21/usr/sbin/munin09:47:450
1360326013,28sleep013660-21/usr/sbin/munin07:57:390
1216126015,24sleep10-21swapper/111:17:381
953025916,22sleep10-21swapper/109:27:361
246482595,10sleep30-21swapper/310:13:363
2451325917,24sleep00-21swapper/008:32:190
1796625918,21sleep011-21ksoftirqd/008:12:200
2890425818,18sleep30-21swapper/310:27:463
917925713,30sleep19183-21ntp_states07:42:461
650325714,24sleep00-21swapper/009:17:370
5432579,14sleep30-21swapper/312:22:403
2906025714,21sleep10-21swapper/112:12:171
257862575,40sleep10-21swapper/108:34:261
2023725717,21sleep30-21swapper/308:17:403
1152825713,23sleep30-21swapper/307:52:173
790225618,21sleep10-21swapper/111:02:481
653825613,25sleep06639-21users10:57:510
4552568,13sleep10-21swapper/112:22:381
1424125615,24sleep314294-21users07:57:503
80682559,24sleep00-21swapper/007:37:480
601925513,24sleep16055-21ntp_states07:32:451
2844525511,20sleep10-21swapper/110:27:351
257825515,24sleep12584-21diskstats07:22:351
2367525515,24sleep023671-21apt-get10:12:280
165912555,11sleep30-21swapper/309:48:013
1483225513,26sleep114869-21latency_hist11:27:191
394425410,30sleep34029-21cat12:32:463
3098725418,19sleep30-21swapper/308:52:333
2283325412,25sleep322828-21apt-get11:52:323
165225414,24sleep01646-21apt-get09:02:310
1035425420,19sleep10-21swapper/107:47:341
535125316,21sleep35371-21ls07:32:183
535125316,21sleep35371-21ls07:32:183
2992025315,20sleep00-21swapper/008:47:460
2692925315,20sleep30-21swapper/308:37:473
2392325313,23sleep123948-21irqstats10:12:371
228092539,17sleep00-21swapper/011:52:290
3176025217,20sleep00-21swapper/010:37:400
2547225213,25sleep125528-21interrupts10:17:381
2165925216,17sleep30-21swapper/308:22:333
880325112,23sleep08808-21diskstats07:42:370
643625117,17sleep30-21swapper/309:17:363
412525111,24sleep04131-21df_inode07:27:350
412525111,24sleep04131-21df_inode07:27:350
357625113,17sleep10-21swapper/109:07:411
3152225113,15sleep00-21swapper/008:52:460
2939525115,15sleep30-21swapper/308:47:283
2786925112,22sleep027864-21apt-get08:42:290
2367025111,16sleep30-21swapper/308:27:463
1516325113,23sleep015255-21perl11:27:380
560025012,22sleep05597-21apt-get07:32:310
415725012,20sleep14271-21users12:32:481
298625013,22sleep03072-21mailstats07:22:460
215052508,16sleep10-21swapper/111:47:381
1077625013,18sleep00-21swapper/009:32:190
2787224912,21sleep327939-21df08:42:323
1974024912,22sleep019734-21apt-get11:42:320
736424810,14sleep00-21swapper/011:02:350
3254924813,20sleep332545-21apt-get08:57:303
3113924815,18sleep10-21swapper/107:12:181
2075824811,22sleep320797-21ntp_states10:02:423
1648624812,21sleep116493-21apt-get08:07:291
1477924812,21sleep314810-21munin-run11:27:173
2333824710,14sleep10-21swapper/111:52:461
1256924710,21sleep012563-21apt-get09:37:320
1411524611,21sleep114112-21apt-get09:42:291
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional