You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-24 - 23:01

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot5s.osadl.org (updated Thu Sep 24, 2026 12:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
999099260,25cyclictest2366-21cron10:35:003
998799250,24cyclictest8643-21/usr/sbin/munin12:38:132
9985992525,0cyclictest0-21swapper/109:37:471
9980992525,0cyclictest0-21swapper/009:37:470
9980992523,1cyclictest0-21swapper/009:13:120
998099250,24cyclictest0-21swapper/011:01:490
999099240,23cyclictest4068-21diskmemload11:31:183
999099240,23cyclictest4068-21diskmemload10:17:273
999099240,23cyclictest4068-21diskmemload09:26:343
9987992423,1cyclictest4068-21diskmemload12:07:252
998799240,23cyclictest1129-21ssh11:32:512
998799240,1cyclictest4068-21diskmemload10:01:432
9985992424,0cyclictest0-21swapper/111:49:001
9985992422,1cyclictest0-21swapper/110:11:071
9985992421,2cyclictest5316-21sh11:07:121
998599240,24cyclictest0-21swapper/112:21:161
998599240,24cyclictest0-21swapper/111:10:291
998599240,23cyclictest21719-21perf08:40:011
998599240,23cyclictest11313-21ssh12:39:261
998599240,1cyclictest4068-21diskmemload10:02:051
9980992424,0cyclictest4068-21diskmemload11:49:000
9980992423,1cyclictest0-21swapper/010:53:420
998099240,23cyclictest0-21swapper/012:21:160
9990992323,0cyclictest0-21swapper/312:05:363
9990992323,0cyclictest0-21swapper/311:22:273
9990992323,0cyclictest0-21swapper/310:50:433
9990992323,0cyclictest0-21swapper/310:48:493
9990992323,0cyclictest0-21swapper/310:48:493
9990992321,1cyclictest10439-21apt-get07:13:053
9990992320,2cyclictest674-21ssh09:35:343
9990992320,2cyclictest3297-21sh10:36:043
9990992320,2cyclictest20120-21ntp_states12:18:173
9990992320,2cyclictest16942-21ssh11:46:393
999099230,2cyclictest313-21systemd-journal09:10:003
999099230,2cyclictest27459-21ssh12:25:003
999099230,2cyclictest11772-21ssh09:16:533
999099230,23cyclictest10103-21ssh11:11:023
999099230,23cyclictest0-21swapper/311:52:023
999099230,23cyclictest0-21swapper/309:52:163
999099230,22cyclictest12923-21ssh09:47:083
999099230,0cyclictest0-21swapper/311:58:233
999099230,0cyclictest0-21swapper/311:26:293
999099230,0cyclictest0-21swapper/310:40:353
999099230,0cyclictest0-21swapper/310:00:483
999099230,0cyclictest0-21swapper/309:13:183
9987992323,0cyclictest0-21swapper/212:31:292
9987992323,0cyclictest0-21swapper/211:38:422
9987992323,0cyclictest0-21swapper/211:26:252
9987992323,0cyclictest0-21swapper/209:25:082
9987992322,1cyclictest4068-21diskmemload10:53:432
9987992321,1cyclictest0-21swapper/209:13:122
998799230,2cyclictest16089-21sh11:45:232
998799230,23cyclictest0-21swapper/211:58:282
998799230,23cyclictest0-21swapper/210:35:332
998799230,22cyclictest7958-21ssh09:42:522
998799230,22cyclictest7226-21ssh10:10:252
998799230,22cyclictest28326-21sh10:29:142
998799230,22cyclictest15050-21sh11:15:162
998799230,22cyclictest13079-21ssh12:12:382
998799230,1cyclictest313-21systemd-journal12:05:002
998799230,0cyclictest4068-21diskmemload11:10:422
998799230,0cyclictest4068-21diskmemload11:04:542
998799230,0cyclictest0-21swapper/211:05:462
998799230,0cyclictest0-21swapper/210:58:232
998799230,0cyclictest0-21swapper/210:15:222
9985992323,0cyclictest0-21swapper/111:28:421
9985992323,0cyclictest0-21swapper/110:53:421
9985992323,0cyclictest0-21swapper/110:45:521
9985992323,0cyclictest0-21swapper/110:45:521
9985992323,0cyclictest0-21swapper/110:36:061
9985992323,0cyclictest0-21swapper/110:21:331
998599232,1cyclictest17560-21sh10:19:391
9985992321,1cyclictest0-21swapper/112:26:161
9985992321,1cyclictest0-21swapper/112:07:201
9985992321,1cyclictest0-21swapper/110:31:471
998599231,21cyclictest0-21swapper/111:45:011
998599230,2cyclictest12024-21sh09:45:491
998599230,23cyclictest0-21swapper/112:14:191
998599230,23cyclictest0-21swapper/111:57:001
998599230,22cyclictest5289-21rm12:33:431
998599230,22cyclictest0-21swapper/112:03:131
998599230,22cyclictest0-21swapper/109:18:041
998599230,0cyclictest0-21swapper/111:00:041
998599230,0cyclictest0-21swapper/109:42:001
998599230,0cyclictest0-21swapper/109:11:031
9980992323,0cyclictest0-21swapper/011:31:060
9980992323,0cyclictest0-21swapper/011:28:420
9980992323,0cyclictest0-21swapper/010:02:550
9980992323,0cyclictest0-21swapper/009:17:470
9980992322,1cyclictest4068-21diskmemload10:58:460
9980992321,1cyclictest2993-21apt-get12:33:050
9980992321,1cyclictest2937-21apt08:03:050
9980992321,1cyclictest0-21swapper/009:22:170
9980992320,2cyclictest17310-21ssh12:15:480
998099230,2cyclictest15577-21munin-plugin-st11:45:010
998099230,23cyclictest4068-21diskmemload12:14:190
998099230,23cyclictest4068-21diskmemload10:39:230
998099230,22cyclictest5290-21sh11:35:500
998099230,22cyclictest4068-21diskmemload12:25:330
998099230,22cyclictest4068-21diskmemload09:59:360
998099230,22cyclictest32238-21systemctl12:00:000
998099230,22cyclictest23526-21apt-get11:53:040
998099230,22cyclictest20141-21idleruntime-cro11:20:000
998099230,22cyclictest10379-21cat12:38:190
998099230,22cyclictest0-21swapper/010:48:070
998099230,22cyclictest0-21swapper/010:48:070
998099230,22cyclictest0-21swapper/009:47:190
998099230,1cyclictest12118-21munin-run08:20:000
9990992222,0cyclictest0-21swapper/310:27:313
9990992222,0cyclictest0-21swapper/310:22:463
9990992222,0cyclictest0-21swapper/309:58:563
9990992221,1cyclictest2906-21ssh10:07:193
9990992220,2cyclictest1103-21sh12:30:253
9990992220,1cyclictest0-21swapper/308:05:003
999099220,2cyclictest6086-21ssh11:37:003
999099220,22cyclictest0-21swapper/312:13:513
999099220,22cyclictest0-21swapper/309:41:073
999099220,21cyclictest7172-21ssh12:36:073
999099220,21cyclictest16885-21cron08:30:013
999099220,21cyclictest0-21swapper/312:02:103
999099220,1cyclictest32719-21ssh11:02:523
999099220,1cyclictest0-21swapper/309:30:263
999099220,0cyclictest0-21swapper/312:26:423
9987992222,0cyclictest4068-21diskmemload11:21:522
9987992222,0cyclictest4068-21diskmemload10:21:122
9987992222,0cyclictest0-21swapper/212:25:092
9987992221,1cyclictest4068-21diskmemload11:44:142
9987992221,1cyclictest0-21swapper/210:06:502
9987992220,1cyclictest396-21ssh09:35:012
9987992220,1cyclictest1298-21munin-run09:05:002
9987992219,2cyclictest23109-21sh11:52:352
998799220,21cyclictest4068-21diskmemload09:20:032
998799220,21cyclictest30656-21ssh10:31:432
998799220,21cyclictest25747-21apt-get07:43:062
998799220,21cyclictest17517-21ssh09:50:422
998799220,21cyclictest12118-21munin-run08:20:002
998799220,21cyclictest0-21swapper/209:05:022
998799220,1cyclictest8997-21sh10:41:072
998799220,1cyclictest11569-21sh09:16:392
998799220,0cyclictest4068-21diskmemload09:46:582
998799220,0cyclictest0-21swapper/210:45:252
998799220,0cyclictest0-21swapper/210:45:252
998799220,0cyclictest0-21swapper/209:30:582
9985992222,0cyclictest0-21swapper/112:17:031
9985992222,0cyclictest0-21swapper/111:31:061
9985992222,0cyclictest0-21swapper/110:40:201
9985992222,0cyclictest0-21swapper/109:53:471
9985992221,1cyclictest0-21swapper/109:27:001
9985992220,1cyclictest25224-21ssh10:55:021
9985992220,1cyclictest10185-21apt-get08:18:061
998599221,20cyclictest1284-21apt-get09:05:021
998599220,22cyclictest0-21swapper/111:53:041
998599220,22cyclictest0-21swapper/109:30:011
998599220,21cyclictest22819-21ssh09:55:221
998599220,21cyclictest0-21swapper/111:15:291
998599220,21cyclictest0-21swapper/110:25:071
998599220,21cyclictest0-21swapper/110:05:051
998599220,1cyclictest17059-21ssh09:21:151
998599220,1cyclictest12840-21apt-get07:18:071
998599220,1cyclictest0-21swapper/111:36:591
9980992222,0cyclictest0-21swapper/010:40:200
9980992222,0cyclictest0-21swapper/010:11:070
9980992221,1cyclictest0-21swapper/010:18:120
9980992221,1cyclictest0-21swapper/009:50:180
9980992221,1cyclictest0-21swapper/009:43:500
9980992220,1cyclictest4478-21ssh11:06:040
9980992220,1cyclictest0-21swapper/011:23:390
998099220,22cyclictest0-21swapper/010:21:160
998099220,22cyclictest0-21swapper/009:30:010
998099220,21cyclictest21803-21ssh09:25:060
998099220,21cyclictest14760-21perf07:20:000
998099220,21cyclictest0-21swapper/012:05:020
998099220,21cyclictest0-21swapper/012:01:290
998099220,21cyclictest0-21swapper/008:53:050
998099220,1cyclictest21724-21awk08:40:010
998099220,1cyclictest0-21swapper/010:27:280
998099220,0cyclictest0-21swapper/010:05:070
9990992121,0cyclictest0-21swapper/311:42:353
9990992121,0cyclictest0-21swapper/311:05:303
9990992121,0cyclictest0-21swapper/310:57:593
9990992120,1cyclictest12603-21apt-get08:23:053
999099210,0cyclictest0-21swapper/310:12:313
9987992120,0cyclictest18042-21apt-get07:28:062
9987992119,1cyclictest31322-21munin-run09:00:002
9987992119,1cyclictest17153-21apt-get07:25:002
998799210,21cyclictest23780-21ssh12:22:092
998799210,21cyclictest0-21swapper/212:15:132
998799210,0cyclictest0-21swapper/208:43:062
998799210,0cyclictest0-21swapper/208:13:042
9985992120,1cyclictest0-21swapper/108:03:051
998599210,21cyclictest0-21swapper/111:21:161
998599210,21cyclictest0-21swapper/107:40:031
998599210,0cyclictest0-21swapper/108:53:051
998599210,0cyclictest0-21swapper/107:48:061
9980992119,1cyclictest4069-21cron09:10:000
9980992119,1cyclictest0-21swapper/007:23:060
9990992020,0cyclictest0-21swapper/309:03:063
9990992020,0cyclictest0-21swapper/307:38:053
9990992020,0cyclictest0-21swapper/307:25:023
999099200,20cyclictest0-21swapper/308:43:073
999099200,0cyclictest0-21swapper/308:53:053
9987992020,0cyclictest0-21swapper/208:48:042
9987992020,0cyclictest0-21swapper/208:33:062
9987992020,0cyclictest0-21swapper/207:18:072
9987992019,1cyclictest19791-21apt-get08:38:072
9987992019,0cyclictest0-21swapper/207:58:052
998799200,20cyclictest0-21swapper/208:08:062
998799200,0cyclictest20439-21apt-get07:33:062
9985992020,0cyclictest0-21swapper/108:28:061
9985992020,0cyclictest0-21swapper/108:23:051
9985992019,1cyclictest5389-21apt-get08:08:061
9980992020,0cyclictest0-21swapper/008:23:050
9980992020,0cyclictest0-21swapper/007:43:060
9980992019,0cyclictest22830-21apt-get07:38:050
9990991919,0cyclictest7779-21apt-get08:13:043
9990991919,0cyclictest17394-21apt-get08:33:053
9990991919,0cyclictest0-21swapper/307:43:063
9987991919,0cyclictest0-21swapper/207:53:052
9985991919,0cyclictest0-21swapper/107:38:051
9985991918,0cyclictest0-21swapper/107:58:051
9985991918,0cyclictest0-21swapper/107:23:061
9980991919,0cyclictest0-21swapper/008:08:060
9980991919,0cyclictest0-21swapper/007:33:060
9990991818,0cyclictest467-21apt-get07:58:053
9990991817,1cyclictest28143-21apt-get07:48:063
9990991816,1cyclictest10185-21apt-get08:18:063
9987991816,1cyclictest24869-21munin-run07:40:012
9987991816,1cyclictest14504-21apt08:25:012
9985991818,0cyclictest0-21swapper/107:33:061
9985991817,0cyclictest0-21swapper/107:28:061
998599180,0cyclictest29389-21apt-get08:58:051
9980991818,0cyclictest0-21swapper/007:58:050
9980991817,1cyclictest24595-21apt-get08:48:040
9990991717,0cyclictest0-21swapper/307:33:063
9990991717,0cyclictest0-21swapper/307:18:073
9990991716,0cyclictest5389-21apt-get08:08:063
9987991716,1cyclictest26995-21apt-get08:53:062
9987991715,1cyclictest622-21gdbus07:50:012
9985991717,0cyclictest0-21swapper/108:48:041
9980991717,0cyclictest0-21swapper/008:28:060
9980991717,0cyclictest0-21swapper/007:28:060
9980991716,1cyclictest0-21swapper/009:03:070
9987991614,1cyclictest12311-21cron07:15:002
9985991616,0cyclictest0-21swapper/108:13:031
9985991614,1cyclictest17394-21apt-get08:33:061
9980991616,0cyclictest0-21swapper/008:33:060
9980991616,0cyclictest0-21swapper/008:13:030
9980991614,1cyclictest30536-21apt-get07:53:050
998099160,1cyclictest0-21swapper/007:15:000
999099157,3cyclictest345-21systemd-udevd08:48:493
9990991515,0cyclictest15252-21apt-get07:23:063
9990991513,1cyclictest0-21swapper/308:58:423
999099150,14cyclictest0-21swapper/307:50:413
9987991515,0cyclictest14997-21apt-get08:28:062
9985991515,0cyclictest0-21swapper/109:03:071
9985991514,1cyclictest0-21swapper/107:53:051
998099153,10cyclictest611-21irqbalance08:57:370
999099140,14cyclictest0-21swapper/308:35:183
998799140,0cyclictest0-21swapper/208:00:492
9985991414,0cyclictest0-21swapper/107:11:061
998599140,0cyclictest0-21swapper/108:40:161
998099140,14cyclictest0-21swapper/007:45:080
998099140,0cyclictest0-21swapper/008:40:160
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional