You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-15 - 12:43

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Mon Jun 15, 2026 00:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1800629748,953sleep230971-1kworker/2:1H21:58:022
8122258516,549sleep28305-21apt-config21:27:362
8122258516,549sleep28305-21apt-config21:27:362
5580237234,245sleep15635-21munin-plugin-st19:37:331
15968237211,287sleep116113-21latency_hist21:52:351
19451232426,229sleep119600-21grep23:43:051
7546232139,221sleep17628-21fschecks_time19:42:541
3177522938,269sleep131847-21mailstats20:58:031
28922228121,63sleep10-21swapper/100:12:581
16279225510,198sleep116429-21grep23:33:041
20227225310,66sleep120370-21grep22:03:061
238692246111,51sleep10-21swapper/123:57:471
13429223719,195sleep013522-21latency_hist20:02:360
4751223017,67sleep04755-21cron21:17:330
28898222945,127sleep10-21swapper/119:08:041
17676222723,91sleep117820-21gpgv21:57:361
31661222238,52sleep10-21swapper/100:22:351
29406221283,111sleep129416-21modprobe20:52:341
29406221283,111sleep129416-21modprobe20:52:341
29084221121,170sleep329221-21sed19:08:073
13081220530,55sleep10-21swapper/121:42:501
4082220341,106sleep14212-21latency_hist19:32:361
9474219330,124sleep19539-21awk19:48:031
9474219330,124sleep19539-21awk19:48:031
5689219115,122sleep10-21swapper/121:18:021
9581219040,58sleep10-21swapper/123:13:011
25421219017,159sleep025460-21ntp_kernel_pll_20:37:580
3207121877,167sleep132124-21basename19:17:541
2399021878,168sleep124047-21cat22:17:361
231021879,139sleep12389-21apt-config19:27:351
23511216634,91sleep123412-21apt-get20:33:191
400921636,145sleep10-21swapper/122:55:531
31043216112,134sleep331062-21find00:18:303
7936215927,117sleep17979-21ntpq23:07:561
26454215913,134sleep126610-21tail22:23:051
3748215823,105sleep10-21swapper/100:36:531
32698215750,45sleep10-21swapper/122:43:001
32698215750,45sleep10-21swapper/122:43:001
26957215631,112sleep127000-21tr20:42:561
1925621559,20sleep10-21swapper/122:02:331
23304215119,109sleep323367-21http23:57:333
17806215150,47sleep10-21swapper/123:38:051
27247214876,53sleep10-21swapper/100:07:571
1702214732,79sleep10-21swapper/122:47:571
11567214610,76sleep10-21swapper/123:18:141
29490214512,117sleep229611-21cat22:33:042
29072214515,115sleep229139-21awk00:13:032
1368214324,79sleep11411-21ntpq00:27:571
27877214125,64sleep127956-21switchtime22:28:061
8410213824,45sleep10-21swapper/121:27:521
8410213824,45sleep10-21swapper/121:27:521
18347213427,35sleep10-21swapper/120:17:361
23443213210,73sleep123525-21apt-config23:57:351
12194213265,35sleep10-21swapper/119:57:531
20743213171,17sleep10-21swapper/123:48:021
2734212910,104sleep02268-21apt-get19:27:380
3040221278,12sleep30-21swapper/319:12:553
3040221278,12sleep30-21swapper/319:12:553
24954212716,87sleep225094-21grep22:18:052
261221227,100sleep12765-21awk21:08:051
261221227,100sleep12765-21awk21:08:051
16968212016,85sleep216488-21apt-get20:12:442
1357211915,19sleep00-21swapper/000:27:560
124921180,7sleep20-21swapper/200:27:532
2483821148,70sleep124953-21sed22:18:041
22067211439,61sleep128139-21cyclictest20:28:101
22067211439,61sleep128139-21cyclictest20:28:101
11897211431,40sleep10-21swapper/121:37:581
11897211431,40sleep10-21swapper/121:37:581
10600211312,53sleep10-21swapper/121:33:051
905211228,51sleep1337-21/usr/sbin/munin21:03:041
30711211214,79sleep230680-21apt-get22:38:042
30711211214,79sleep230680-21apt-get22:38:042
2224921124,94sleep20-21swapper/222:11:142
2155921118,65sleep121406-21latency_hist20:27:371
2155921118,65sleep121406-21latency_hist20:27:371
16521211117,56sleep116607-21apt-config20:12:351
30064211039,34sleep10-21swapper/120:52:581
30064211039,34sleep10-21swapper/120:52:581
18471211020,76sleep318433-21latency_hist23:42:353
2809210916,76sleep32877-21ls00:32:553
23838210919,69sleep223325-21apt-get23:57:442
16842210912,79sleep216902-21modprobe23:37:332
18898210813,77sleep218956-21/usr/sbin/munin23:42:542
1256821087,61sleep10-21swapper/123:22:511
679921078,65sleep16830-21sort21:22:551
679921078,65sleep16830-21sort21:22:551
28097210736,54sleep128241-21latency_hist19:07:361
4410210522,62sleep24465-21fschecks_count19:32:532
26587210520,10sleep10-21swapper/100:06:201
26248210516,74sleep226310-21sendmail_mailqu22:23:022
740210416,64sleep0337-21/usr/sbin/munin21:02:590
29140210414,73sleep329264-21sed00:13:043
22351210410,78sleep022398-21systemd-run22:12:340
383210312,72sleep0460-21/usr/sbin/munin22:43:030
383210312,72sleep0460-21/usr/sbin/munin22:43:030
11803210314,76sleep011896-21cut21:37:570
11803210314,76sleep011896-21cut21:37:570
3973210112,73sleep24175-21apt-key00:37:352
26756210122,53sleep026899-21latency_hist00:07:360
661829916,27sleep00-21swapper/021:22:500
661829916,27sleep00-21swapper/021:22:500
2708429918,64sleep327167-21wc22:27:353
716829821,13sleep10-21swapper/119:42:341
716829821,13sleep10-21swapper/119:42:341
445829838,43sleep10-21swapper/122:57:471
3172929819,60sleep30-21swapper/320:58:013
980029719,52sleep09948-21grep23:13:040
1316729624,48sleep213323-21grep23:23:042
889629514,68sleep38926-21proc_pri21:28:223
889629514,68sleep38926-21proc_pri21:28:223
36829520,24sleep30-21swapper/321:02:493
2198829510,71sleep122002-21/usr/sbin/munin22:08:061
1502029519,28sleep30-21swapper/320:07:363
59692948,12sleep00-21swapper/021:18:050
45462935,77sleep10-21swapper/121:14:551
274329312,68sleep32757-21acpi19:27:453
190692939,48sleep119126-21cut20:18:081
1826829321,51sleep318287-21cron20:17:343
71042927,73sleep10-21swapper/123:05:221
321652927,73sleep132183-21grep00:22:561
321652927,73sleep132183-21grep00:22:561
3198429123,39sleep232023-21fschecks_count00:22:532
3198429123,39sleep232023-21fschecks_count00:22:532
116302917,34sleep10-21swapper/119:55:261
116302917,34sleep10-21swapper/119:55:261
1052729115,61sleep010597-21switchtime21:33:040
2750629014,59sleep327537-21idleruntime22:27:563
1415329037,38sleep10-21swapper/120:03:061
1166229021,49sleep211773-21tail23:18:152
971528913,61sleep39792-21timerandwakeup23:13:033
2961028916,20sleep20-21swapper/219:12:342
160228918,48sleep31310-21/usr/sbin/munin22:47:553
518728823,40sleep25327-21unixbench_singl22:58:042
255972889,66sleep125677-21sed20:38:041
318952875,65sleep00-21swapper/019:17:490
2463228613,52sleep324771-21/usr/sbin/munin23:58:103
238102869,60sleep222816-21kworker/2:220:33:322
1355528513,51sleep213616-21sendmail_mailqu21:43:032
3216328424,29sleep30-21swapper/319:17:563
2982828417,46sleep029896-21unin-run00:17:340
875228315,52sleep28828-21sed19:47:362
715428329,33sleep07165-21modprobe19:42:330
715428329,33sleep07165-21modprobe19:42:330
488328316,45sleep35025-21latency_hist21:17:363
2977228329,17sleep129778-21munin-node19:12:371
2977228329,17sleep129778-21munin-node19:12:371
2727028326,33sleep027413-21unixbench_multi20:43:030
2630128324,37sleep226311-21modprobe20:42:332
732628216,50sleep07468-21latency_hist23:07:350
163028216,45sleep21651-21ls21:07:332
1191328222,36sleep211957-21ntpq21:37:592
1191328222,36sleep211957-21ntpq21:37:592
2887628148,16sleep10-21swapper/122:32:461
1853228112,17sleep00-21swapper/023:42:360
1364128116,49sleep213659-21/usr/sbin/munin20:02:462
1537428022,16sleep10-21swapper/123:32:361
3091027923,38sleep10-21swapper/122:38:161
3091027923,38sleep10-21swapper/122:38:161
1257927823,26sleep30-21swapper/319:58:033
483027719,45sleep24779-21latency_hist21:17:352
2052927712,50sleep320623-21memory23:47:563
2058927618,42sleep220596-21ntp_states20:23:062
2058927618,42sleep220596-21ntp_states20:23:062
33327513,47sleep0349-21vmstat00:23:070
33327513,47sleep0349-21vmstat00:23:070
2383027512,54sleep217331-21kworker/2:222:16:322
2082727516,37sleep020956-21grep20:23:120
2082727516,37sleep020956-21grep20:23:120
2397927413,45sleep0302-21systemd-logind22:17:350
172862745,11sleep00-21swapper/021:55:410
59592730,7sleep30-21swapper/323:02:363
240427319,27sleep20-21swapper/219:27:362
2341527218,25sleep30-21swapper/320:32:483
2341527218,25sleep30-21swapper/320:32:483
2514027117,25sleep30-21swapper/320:37:533
2080427117,37sleep020815-21modprobe22:07:330
1100827111,24sleep30-21swapper/319:53:023
1100827111,24sleep30-21swapper/319:53:023
380227013,40sleep03873-21http00:37:330
3127327015,24sleep00-21swapper/020:57:500
76912698,13sleep30-21swapper/319:42:563
352326916,39sleep33645-21sed22:53:043
3105426922,23sleep20-21swapper/220:57:352
3105426922,23sleep20-21swapper/220:57:352
2802426920,28sleep328052-21munin-run19:07:343
2713426932,23sleep20-21swapper/219:02:572
2704126920,25sleep30-21swapper/320:42:583
2307126924,27sleep20-21swapper/220:32:352
2307126924,27sleep20-21swapper/220:32:352
1692326914,28sleep20-21swapper/221:53:052
1647226922,22sleep00-21swapper/020:12:340
30426815,33sleep30-21swapper/322:43:023
30426815,33sleep30-21swapper/322:43:023
214126812,43sleep32198-21awk21:07:543
214126812,43sleep32198-21awk21:07:543
609926715,30sleep20-21swapper/223:02:522
165826716,20sleep00-21swapper/019:23:020
111882678,12sleep00-21swapper/023:18:050
1041526722,29sleep20-21swapper/221:33:022
907626614,29sleep30-21swapper/319:47:533
907626614,29sleep30-21swapper/319:47:533
712326622,27sleep30-21swapper/321:23:053
712326622,27sleep30-21swapper/321:23:053
2553626620,24sleep00-21swapper/022:22:350
2153726624,26sleep021584-21unin-run23:52:330
887426519,30sleep38987-21latency_hist23:12:353
2585826516,27sleep00-21swapper/022:22:520
2372726513,35sleep023784-21/usr/sbin/munin20:33:300
2294126518,31sleep022977-21http20:32:330
2294126518,31sleep022977-21http20:32:330
1867326511,39sleep018699-21fschecks_count20:17:550
1798326521,25sleep30-21swapper/321:57:463
1611726511,29sleep30-21swapper/323:33:023
53822645,15sleep00-21swapper/022:58:140
2945726416,28sleep30-21swapper/320:52:363
2945726416,28sleep30-21swapper/320:52:363
2732526411,23sleep20-21swapper/222:27:502
2034726419,22sleep30-21swapper/320:22:593
2034726419,22sleep30-21swapper/320:22:593
2842126311,36sleep028515-21memory20:47:560
2842126311,36sleep028515-21memory20:47:560
2743626317,25sleep20-21swapper/200:08:022
971726212,36sleep09777-21latency_hist21:32:350
971726212,36sleep09777-21latency_hist21:32:350
720426213,23sleep20-21swapper/219:42:352
720426213,23sleep20-21swapper/219:42:352
566026217,20sleep00-21swapper/019:37:360
311832627,11sleep00-21swapper/022:38:230
311832627,11sleep00-21swapper/022:38:230
279952625,18sleep00-21swapper/019:07:110
1842026214,34sleep218470-21unin-run23:42:342
1636726211,20sleep30-21swapper/321:52:543
3260726117,26sleep20-21swapper/222:42:572
3260726117,26sleep20-21swapper/222:42:572
1489226113,31sleep214908-21/usr/sbin/munin21:47:562
1047326117,26sleep20-21swapper/223:17:352
680826018,19sleep30-21swapper/319:38:083
680826018,19sleep30-21swapper/319:38:083
3040426012,23sleep00-21swapper/020:53:050
3040426012,23sleep00-21swapper/020:53:050
2969426016,27sleep329689-21apt-get20:52:483
2969426016,27sleep329689-21apt-get20:52:483
259226015,30sleep02611-21munin-run22:52:320
239826014,24sleep20-21swapper/221:08:012
239826014,24sleep20-21swapper/221:08:012
2264126017,21sleep30-21swapper/322:12:493
1412126015,19sleep00-21swapper/023:27:500
1284226013,33sleep312975-21sed21:42:363
1284226013,33sleep312975-21sed21:42:363
110272606,11sleep30-21swapper/321:35:093
700225914,24sleep30-21swapper/323:04:073
174302598,12sleep00-21swapper/023:37:560
1508025915,29sleep315086-21ntp_states21:48:013
1368725916,25sleep013788-21/usr/sbin/munin21:43:050
3079825816,22sleep00-21swapper/019:13:040
3079825816,22sleep00-21swapper/019:13:040
2884025813,29sleep028897-21/usr/sbin/munin19:08:030
264425813,31sleep22723-21latency_hist22:52:352
224225816,26sleep32275-21awk19:27:333
1862625813,21sleep00-21swapper/021:58:240
1094525815,27sleep010987-21/usr/sbin/munin19:52:590
1094525815,27sleep010987-21/usr/sbin/munin19:52:590
954125713,19sleep20-21swapper/219:48:042
954125713,19sleep20-21swapper/219:48:042
506525716,25sleep35062-21apt-get21:17:443
3189225713,28sleep20-21swapper/219:17:472
2814025717,24sleep228136-21apt-get20:47:462
2814025717,24sleep228136-21apt-get20:47:462
2477425712,19sleep00-21swapper/023:58:110
2337325715,26sleep023370-40mandb23:57:340
160325719,20sleep30-21swapper/300:28:033
151612575,11sleep30-21swapper/323:30:483
39722565,11sleep20-21swapper/219:31:482
3106625612,24sleep00-21swapper/000:18:310
2739625614,26sleep027428-21ntp_states00:08:000
156152569,34sleep00-21swapper/021:49:070
1521325613,27sleep315279-21cpuspeed20:07:483
1493325614,22sleep20-21swapper/223:28:052
1349425616,25sleep313531-21ntp_states21:43:003
1248725612,27sleep312556-21df23:22:493
899225512,24sleep09063-21df19:47:500
899225512,24sleep09063-21df19:47:500
800125514,22sleep00-21swapper/019:43:050
3095225519,20sleep20-21swapper/219:13:062
3095225519,20sleep20-21swapper/219:13:062
2732325512,25sleep027318-21apt-get22:27:480
1050825520,18sleep20-21swapper/219:52:452
1050825520,18sleep20-21swapper/219:52:452
2935825415,18sleep00-21swapper/022:33:020
2025525413,24sleep220251-21apt-get23:47:472
1883525415,24sleep218864-21irqstats20:18:012
1754025415,19sleep00-21swapper/020:13:590
1558525414,23sleep015579-21apt-get23:32:470
1521225413,24sleep215206-21apt-get20:07:472
148192546,37sleep00-21swapper/020:06:120
29352534,15sleep00-21swapper/021:09:170
29352534,15sleep00-21swapper/021:09:170
2470925316,19sleep30-21swapper/322:18:023
765325215,21sleep07743-21perl23:07:510
2286225212,23sleep222960-21users23:53:162
683125113,21sleep26884-21irqstats21:22:562
683125113,21sleep26884-21irqstats21:22:562
513225112,15sleep30-21swapper/319:33:073
3153825115,16sleep20-21swapper/220:57:562
3068325112,22sleep330153-21apt-get22:37:453
3068325112,22sleep330153-21apt-get22:37:453
301082515,37sleep30-21swapper/322:37:133
127062519,15sleep00-21swapper/023:22:540
515725013,22sleep25214-21fschecks_time21:17:512
425225017,17sleep20-21swapper/222:57:352
297925011,22sleep03088-21perl22:52:530
2681125012,22sleep226856-21irqstats20:42:532
2623225011,14sleep30-21swapper/300:03:073
1886525013,24sleep30-21swapper/320:18:023
809524911,15sleep20-21swapper/223:08:012
1208524910,18sleep20-21swapper/219:57:452
3226624810,19sleep30-21swapper/300:22:583
3226624810,19sleep30-21swapper/300:22:583
3031524812,22sleep229815-21apt-get00:17:402
2787224816,16sleep30-21swapper/300:08:073
1090524811,16sleep30-21swapper/323:17:443
2431924711,22sleep324417-21users20:33:413
1954924611,15sleep20-21swapper/222:02:502
2540524510,20sleep225417-21apt-get00:02:452
213642458,14sleep30-21swapper/322:07:563
1870824511,17sleep30-21swapper/323:42:473
1954524212,16sleep30-21swapper/322:02:473
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional