You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-07 - 18:21

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Mon Sep 07, 2026 12:43:53)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
6703220488,2023sleep027075-1kworker/0:0H07:33:460
2136242957,314sleep120310-21kworker/u8:010:43:231
17181240210,377sleep117384-21sed09:48:261
8081239315,363sleep28125-21ntpq09:18:492
7401234641,292sleep17459-21unin-run09:18:241
6659230845,246sleep16676-21apt-get07:33:351
3461230515,225sleep13531-21switchtime12:28:531
3117322619,238sleep031183-21ntpq07:08:520
3117322619,238sleep031183-21ntpq07:08:520
27700226069,93sleep127779-21threads10:18:531
13682224977,23sleep10-21swapper/109:38:251
5271220310,125sleep15289-21latency_hist10:53:241
5271220310,125sleep15289-21latency_hist10:53:241
15052218712,143sleep10-21swapper/107:58:411
1114218214,88sleep117-21ksoftirqd/112:23:241
282121778,154sleep12718-21cron09:03:261
30862216911,145sleep130904-21cat12:13:431
30862216911,145sleep130904-21cat12:13:431
3122021689,84sleep131357-21grep08:48:581
3202221678,82sleep10-21swapper/110:33:471
26230216710,142sleep126278-21ntpq08:33:481
26230216710,142sleep126278-21ntpq08:33:481
390821669,73sleep13985-21switchtime07:23:541
16221216517,130sleep316377-21latency_hist11:28:253
16221216517,130sleep316377-21latency_hist11:28:253
2566216418,66sleep10-21swapper/107:18:581
21475216415,29sleep00-21swapper/008:18:460
18685216265,19sleep10-21swapper/108:08:541
32457216143,106sleep132493-21ntpq08:53:501
517621609,118sleep10-21swapper/107:28:461
24203215816,91sleep124206-21latency_hist11:53:251
1205921556,11sleep10-21swapper/107:48:481
12353215231,69sleep112445-21cat11:13:541
14229215012,82sleep10-21swapper/111:18:551
14229215012,82sleep10-21swapper/111:18:551
1070921509,57sleep10-21swapper/111:08:541
1070921509,57sleep10-21swapper/111:08:541
724214518,74sleep10-21swapper/107:13:531
724214518,74sleep10-21swapper/107:13:531
5062214350,80sleep15156-21timerandwakeup12:33:531
7470213919,22sleep00-21swapper/010:58:480
7470213919,22sleep00-21swapper/010:58:480
30812213327,17sleep10-21swapper/107:08:441
30812213327,17sleep10-21swapper/107:08:441
29229212712,97sleep10-21swapper/112:08:431
8727212541,69sleep10-21swapper/111:03:411
8727212541,69sleep10-21swapper/111:03:411
31193212516,92sleep231253-21/usr/sbin/munin07:08:542
31193212516,92sleep231253-21/usr/sbin/munin07:08:542
14123212421,81sleep314269-21unixbench_singl07:53:553
1418212243,62sleep1922-21apt-get08:58:341
7738212011,92sleep27841-21timerwakeupswit10:58:542
7738212011,92sleep27841-21timerwakeupswit10:58:542
59721200,11sleep2584-21munin-run10:38:232
17467212039,61sleep10-21swapper/109:48:341
17467212039,61sleep10-21swapper/109:48:341
17723211943,64sleep117736-21munin-run11:33:231
17975211819,77sleep318030-21cpuspeed11:33:383
21205211650,54sleep121180-21/usr/sbin/munin08:18:391
10259211617,78sleep09779-21apt-get07:43:350
28920211549,50sleep10-21swapper/110:23:461
10384211417,73sleep310420-21/usr/sbin/munin07:43:463
21909211247,53sleep121955-21unin-run10:03:241
30565211138,52sleep10-21swapper/110:28:461
8415210937,54sleep10-21swapper/111:03:251
8415210937,54sleep10-21swapper/111:03:251
3383210913,17sleep20-21swapper/209:03:482
3136121080,8sleep20-21swapper/208:48:582
29903210822,21sleep20-21swapper/210:28:232
26105210716,79sleep126139-21cut11:58:421
26105210716,79sleep126139-21cut11:58:421
255721078,88sleep00-21swapper/009:01:200
25560210736,14sleep10-21swapper/108:33:241
25560210736,14sleep10-21swapper/108:33:241
22095210719,61sleep322262-21latency_hist10:03:263
15001210712,73sleep10-21swapper/111:23:431
15001210712,73sleep10-21swapper/111:23:431
8474210572,20sleep18493-21apt-get07:38:351
6679210513,20sleep20-21swapper/207:33:362
23052210535,52sleep10-21swapper/108:23:471
20073210534,14sleep10-21swapper/108:13:511
20073210534,14sleep10-21swapper/108:13:511
28563210435,23sleep10-21swapper/112:04:511
3818210315,73sleep33839-21awk07:23:523
9649210212,73sleep19694-21ntpq09:23:481
9649210212,73sleep19694-21ntpq09:23:481
5952210234,13sleep10-21swapper/110:53:481
5952210234,13sleep10-21swapper/110:53:481
10293210235,14sleep10-21swapper/107:43:401
27865210134,51sleep10-21swapper/108:38:531
27865210134,51sleep10-21swapper/108:38:531
16966210119,65sleep216977-21modprobe09:48:232
16966210119,65sleep216977-21modprobe09:48:232
15235210116,23sleep00-21swapper/009:43:250
15235210116,23sleep00-21swapper/009:43:250
9011210020,62sleep39138-21latency_hist09:23:263
18139210015,67sleep318146-21/usr/sbin/munin08:08:403
11238210032,15sleep10-21swapper/109:28:491
269629923,59sleep32761-21grep10:43:453
269629923,59sleep32761-21grep10:43:453
2238029931,14sleep10-21swapper/110:03:281
2238029931,14sleep10-21swapper/110:03:281
2134929910,11sleep20-21swapper/211:43:422
1713629918,17sleep10-21swapper/108:03:541
1620329911,68sleep016269-21mailstats09:44:010
1620329911,68sleep016269-21mailstats09:44:010
3235329815,64sleep032402-21fschecks_count12:18:430
1816029721,51sleep218249-21fschecks_time08:08:432
1532229756,22sleep10-21swapper/109:43:261
1532229756,22sleep10-21swapper/109:43:261
2030329630,13sleep10-21swapper/109:58:231
2030329630,13sleep10-21swapper/109:58:231
25742957,13sleep10-21swapper/110:43:431
25742957,13sleep10-21swapper/110:43:431
2405529415,53sleep024088-21fschecks_count10:08:430
312002938,76sleep027075-1kworker/0:0H10:29:580
218729319,21sleep30-21swapper/310:43:253
113029329,31sleep20-21swapper/212:23:252
540129210,62sleep10-21swapper/109:08:551
295982927,11sleep30-21swapper/307:03:503
197522929,70sleep319811-21if_eth011:38:463
185532928,18sleep10-21swapper/111:33:521
174229260,16sleep10-21swapper/112:23:481
2766229023,45sleep227726-21fschecks_time12:03:432
1244629013,21sleep20-21swapper/211:13:552
380928913,62sleep03823-21vmstat12:28:560
2131228915,57sleep320831-21apt-get11:43:343
677928825,31sleep30-21swapper/310:58:223
677928825,31sleep30-21swapper/310:58:223
2485228814,58sleep024894-21ntp_states11:53:490
1243528815,56sleep20-21swapper/209:33:462
65128716,45sleep00-21swapper/010:38:260
395628712,58sleep00-21swapper/010:48:390
395628712,58sleep00-21swapper/010:48:390
487428631,29sleep20-21swapper/207:28:352
247722867,14sleep10-21swapper/110:08:571
1988028617,48sleep019894-21grep08:13:450
1988028617,48sleep019894-21grep08:13:450
3093428511,57sleep330988-21interrupts07:08:463
3093428511,57sleep330988-21interrupts07:08:463
709928416,54sleep37127-21cpu10:58:393
709928416,54sleep37127-21cpu10:58:393
2427528417,35sleep024294-21apt-get08:28:340
2427528417,35sleep024294-21apt-get08:28:340
162702849,17sleep10-21swapper/109:44:021
2962728314,54sleep229719-21mailstats12:08:522
2094428326,31sleep20-21swapper/211:43:252
318028216,47sleep23256-21latency12:28:452
213202829,60sleep121339-21apt-get11:43:361
244392817,15sleep10-21swapper/111:53:371
1961528111,39sleep30-21swapper/308:13:363
1961528111,39sleep30-21swapper/308:13:363
1868428112,56sleep2367-21snmpd09:52:272
1868428112,56sleep2367-21snmpd09:52:272
1660928115,42sleep216672-21cut08:03:432
1020228117,30sleep30-21swapper/311:08:403
1020228117,30sleep30-21swapper/311:08:403
849728015,50sleep38502-21modprobe07:38:373
264572805,65sleep10-21swapper/110:14:211
1379828012,14sleep10-21swapper/107:53:491
301212795,27sleep10-21swapper/108:46:561
2965627919,38sleep229813-21tail08:43:542
2628627921,29sleep00-21swapper/011:58:450
2628627921,29sleep00-21swapper/011:58:450
2913027824,33sleep129127-21apt-get07:03:361
286962789,12sleep30-21swapper/310:23:403
28422789,14sleep20-21swapper/210:43:492
28422789,14sleep20-21swapper/210:43:492
275527812,52sleep32764-21systemd-run09:03:243
923827712,50sleep20-21swapper/211:03:542
923827712,50sleep20-21swapper/211:03:542
507427714,47sleep25167-21mailstats09:08:522
195572778,17sleep10-21swapper/111:38:401
1275127715,27sleep00-21swapper/009:33:520
933527625,29sleep30-21swapper/309:23:423
933527625,29sleep30-21swapper/309:23:423
3275727616,37sleep3305-21fschecks_count07:13:433
3275727616,37sleep3305-21fschecks_count07:13:433
2840327612,49sleep028442-21cat10:23:240
88127521,27sleep20-21swapper/210:38:402
325927516,23sleep30-21swapper/312:28:473
2623927521,32sleep226382-21grep10:13:542
89332748,16sleep1367-21snmpd09:23:041
2778127418,29sleep20-21swapper/210:18:542
2940427325,29sleep20-21swapper/207:03:452
2282427310,30sleep30-21swapper/310:03:503
2282427310,30sleep30-21swapper/310:03:503
529027219,28sleep20-21swapper/210:53:252
529027219,28sleep20-21swapper/210:53:252
30562729,11sleep20-21swapper/207:23:182
2288927216,38sleep222954-21awk10:03:532
2288927216,38sleep222954-21awk10:03:532
2287427214,40sleep222960-21/usr/sbin/munin08:23:442
1790427215,29sleep30-21swapper/309:49:173
1790427215,29sleep30-21swapper/309:49:173
68512718,42sleep00-21swapper/009:13:550
2747027115,38sleep327498-21/usr/sbin/munin10:18:473
2678327117,27sleep00-21swapper/010:18:240
264112710,7sleep20-21swapper/208:33:522
264112710,7sleep20-21swapper/208:33:522
204827115,39sleep32082-21/usr/sbin/munin07:18:473
1077027123,25sleep30-21swapper/309:28:353
2944027013,40sleep329480-21/usr/sbin/munin08:43:513
2872327012,26sleep00-21swapper/010:23:410
2570027012,43sleep225701-21cron11:58:232
2173827010,40sleep30-21swapper/308:18:533
186827017,32sleep21903-21/usr/sbin/munin08:59:242
1213527011,42sleep212286-21cat09:33:262
309052690,9sleep219189-21kworker/2:012:13:432
309052690,9sleep219189-21kworker/2:012:13:432
2167226914,34sleep221737-21awk08:18:522
2071926921,33sleep220811-21sort09:58:452
1360126915,40sleep313651-21modprobe09:38:233
98826820,28sleep00-21swapper/008:58:250
239112684,11sleep20-21swapper/211:50:152
2328526813,41sleep323357-21sensors08:23:533
208426813,24sleep00-21swapper/007:18:480
200672678,10sleep00-21swapper/009:55:200
200672678,10sleep00-21swapper/009:55:200
199832675,10sleep30-21swapper/309:54:183
199832675,10sleep30-21swapper/309:54:183
1651526720,19sleep30-21swapper/311:28:413
89126619,31sleep2892-21cron08:58:232
70722667,12sleep30-21swapper/307:33:573
2789626617,22sleep30-21swapper/308:38:553
2789626617,22sleep30-21swapper/308:38:553
252782666,51sleep224015-21kworker/u8:008:29:462
252782666,51sleep224015-21kworker/u8:008:29:462
2423526610,25sleep30-21swapper/310:08:463
52992657,12sleep30-21swapper/307:28:483
3036726514,28sleep20-21swapper/210:28:432
268672658,42sleep00-21swapper/008:35:160
268672658,42sleep00-21swapper/008:35:160
2415226524,26sleep324172-21modprobe11:53:233
183742656,13sleep00-21swapper/011:33:470
1750326514,31sleep017469-21/usr/sbin/munin09:48:420
1750326514,31sleep017469-21/usr/sbin/munin09:48:420
445726413,30sleep34431-21/usr/sbin/munin12:33:393
99326319,20sleep00-21swapper/007:13:560
99326319,20sleep00-21swapper/007:13:560
3062226313,21sleep00-21swapper/008:48:440
2598926318,26sleep30-21swapper/310:13:513
2239026318,21sleep00-21swapper/010:03:350
2239026318,21sleep00-21swapper/010:03:350
155132637,11sleep30-21swapper/307:58:513
1119326318,28sleep00-21swapper/009:28:480
465626224,19sleep00-21swapper/009:08:420
3253426221,19sleep30-21swapper/312:18:463
317932628,35sleep30-21swapper/308:53:233
1927226222,23sleep00-21swapper/008:12:400
1428626217,28sleep214312-21netstat09:38:472
1428626217,28sleep214312-21netstat09:38:472
591726115,21sleep00-21swapper/010:53:470
591726115,21sleep00-21swapper/010:53:470
2980326122,19sleep00-21swapper/012:08:540
200626112,30sleep32134-21/usr/sbin/munin12:23:543
1514926116,21sleep20-21swapper/207:58:422
1312226112,17sleep00-21swapper/011:18:250
114502616,21sleep00-21swapper/011:13:170
114502616,21sleep00-21swapper/011:13:170
3103026017,20sleep00-21swapper/012:13:460
3103026017,20sleep00-21swapper/012:13:460
2937526020,22sleep00-21swapper/008:43:480
178826017,25sleep21817-21ntp_states12:23:492
143882606,11sleep00-21swapper/011:20:430
143882606,11sleep00-21swapper/011:20:430
1221826014,28sleep012248-21/usr/sbin/munin07:48:510
590125916,21sleep30-21swapper/309:13:253
47825911,25sleep20-21swapper/207:13:462
47825911,25sleep20-21swapper/207:13:462
454325913,27sleep24579-21/usr/sbin/munin12:33:422
315625915,21sleep00-21swapper/010:43:550
315625915,21sleep00-21swapper/010:43:550
1441725913,20sleep30-21swapper/309:38:523
1441725913,20sleep30-21swapper/309:38:523
85292587,40sleep00-21swapper/007:38:540
770125815,27sleep07695-21apt-get09:18:390
619425812,30sleep26319-21sed10:53:532
619425812,30sleep26319-21sed10:53:532
1961925810,12sleep20-21swapper/208:13:392
1961925810,12sleep20-21swapper/208:13:392
17292587,12sleep00-21swapper/012:23:470
148532588,13sleep00-21swapper/011:23:340
148532588,13sleep00-21swapper/011:23:340
1465025817,26sleep014711-21sendmail-msp07:58:240
1340425819,22sleep213411-21sshd11:18:392
1340425819,22sleep213411-21sshd11:18:392
1214925815,27sleep312216-21memory07:48:503
112172585,11sleep20-21swapper/211:10:162
112172585,11sleep20-21swapper/211:10:162
533425713,26sleep05357-21ntp_states07:28:490
3235725721,19sleep30-21swapper/308:53:483
2527625716,17sleep20-21swapper/210:13:262
2188725719,21sleep00-21swapper/010:03:230
1953025716,25sleep019549-21apt-get11:38:350
2757325616,25sleep027568-21apt-get12:03:370
268012566,11sleep30-21swapper/308:34:273
268012566,11sleep30-21swapper/308:34:273
1768625614,27sleep017708-21munin-run11:33:220
1496425614,25sleep014957-21apt-get07:58:360
1360325617,21sleep30-21swapper/311:18:443
1360325617,21sleep30-21swapper/311:18:443
1221525615,23sleep312279-21ps11:13:523
406025518,22sleep34067-21/usr/sbin/munin10:48:463
406025518,22sleep34067-21/usr/sbin/munin10:48:463
2647625516,23sleep226482-21ntp_states11:58:512
2647625516,23sleep226482-21ntp_states11:58:512
2310225511,27sleep023142-21if_eth011:48:440
1964525516,21sleep20-21swapper/209:53:522
1964525516,21sleep20-21swapper/209:53:522
635725415,24sleep26386-21irqstats09:13:452
611425412,25sleep36109-21apt-get09:13:373
445425413,25sleep04449-21apt-get12:33:360
3043425416,19sleep30-21swapper/310:28:443
2910825414,20sleep00-21swapper/007:03:350
2275025415,23sleep022744-21apt-get08:23:380
373025316,22sleep00-21swapper/007:23:490
3222725316,20sleep30-21swapper/310:33:533
293832538,30sleep30-21swapper/312:08:453
2457325314,21sleep324667-21memory08:28:463
2457325314,21sleep324667-21memory08:28:463
1864625312,25sleep218776-21sed11:33:532
1697925312,27sleep317044-21dpkg09:48:243
1697925312,27sleep317044-21dpkg09:48:243
1618125314,21sleep20-21swapper/211:28:242
1618125314,21sleep20-21swapper/211:28:242
979925210,17sleep20-21swapper/207:43:242
936025211,17sleep00-21swapper/009:23:430
936025211,17sleep00-21swapper/009:23:430
874225212,19sleep30-21swapper/311:03:433
874225212,19sleep30-21swapper/311:03:433
87925014,24sleep0874-21apt-get10:38:380
865025012,22sleep08647-21apt-get11:03:360
865025012,22sleep08647-21apt-get11:03:360
364825013,20sleep00-21swapper/009:03:540
1642725011,23sleep216423-21apt-get11:28:362
1314725014,17sleep20-21swapper/207:53:252
398724918,17sleep20-21swapper/207:23:552
300324911,22sleep32999-21apt-get09:03:373
214242499,15sleep00-21swapper/011:43:440
2732524813,20sleep31-21systemd12:03:243
2732524813,20sleep31-21systemd12:03:243
2571324816,17sleep30-21swapper/311:58:243
1783324811,19sleep00-21swapper/008:08:250
175412485,33sleep30-21swapper/308:05:173
1029424811,21sleep210278-21apt-get07:43:412
1384624713,19sleep213852-21ntp_states07:53:502
932724613,20sleep29334-21diskstats09:23:412
932724613,20sleep29334-21diskstats09:23:412
283902465,32sleep20-21swapper/208:39:342
283902465,32sleep20-21swapper/208:39:342
3094824511,14sleep30-21swapper/312:13:453
3094824511,14sleep30-21swapper/312:13:453
284402418,16sleep30-21swapper/312:03:573
2868523911,14sleep00-21swapper/008:43:240
2868523911,14sleep00-21swapper/008:43:240
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional