You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-06 - 01:32

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Sun Jul 05, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3055329078,8sleep00-21swapper/007:06:040
3061228961,8sleep20-21swapper/207:06:512
3071428063,13sleep10-21swapper/107:08:101
2919627861,16sleep30-21swapper/307:05:153
3094699260,2cyclictest0-21swapper/112:23:101
3094899250,25cyclictest0-21swapper/209:14:572
3094699250,2cyclictest0-21swapper/111:02:461
30951992424,0cyclictest0-21swapper/312:39:233
30948992423,1cyclictest0-21swapper/210:44:112
30948992422,1cyclictest31612-21ssh11:09:432
3094199241,23cyclictest0-21swapper/010:24:020
30951992318,1cyclictest0-21swapper/311:12:063
3094899220,21cyclictest0-21swapper/210:35:132
30951992120,1cyclictest0-21swapper/309:01:043
30951992118,1cyclictest0-21swapper/311:09:453
30948992119,1cyclictest0-21swapper/210:15:372
3094899210,21cyclictest0-21swapper/210:14:182
3094899210,20cyclictest0-21swapper/211:04:352
3094899210,19cyclictest0-21swapper/208:10:222
3094699210,2cyclictest0-21swapper/111:05:151
3094699210,20cyclictest0-21swapper/111:36:331
3094899206,14cyclictest0-21swapper/208:36:312
3094899206,14cyclictest0-21swapper/208:36:312
3094899200,1cyclictest0-21swapper/210:57:322
30946992016,4cyclictest23520-21diskmemload09:51:581
30948991917,1cyclictest6541-21ssh09:18:542
3094899190,18cyclictest0-21swapper/211:41:432
3094899190,0cyclictest0-21swapper/212:35:312
30941991910,5cyclictest18425-21apache_processe11:40:120
30948991817,1cyclictest0-21swapper/211:25:142
3094899180,0cyclictest0-21swapper/210:27:042
3094699182,1cyclictest26130-21diskstats11:25:151
3094699182,12cyclictest0-21swapper/110:39:361
37212170,0sleep30-21swapper/312:09:543
3095199170,16cyclictest0-21swapper/312:04:253
3095199170,16cyclictest0-21swapper/312:04:253
30948991716,1cyclictest0-21swapper/209:25:132
30948991715,1cyclictest2702-21ssh11:50:072
30948991714,1cyclictest0-21swapper/212:18:252
3094199176,10cyclictest0-21swapper/009:05:150
3094199173,7cyclictest16509-21basename11:20:000
3095199160,1cyclictest0-21swapper/309:54:443
3095199160,16cyclictest0-21swapper/310:43:263
30948991615,1cyclictest0-21swapper/211:35:242
30948991615,1cyclictest0-21swapper/209:43:542
3094899160,16cyclictest0-21swapper/211:34:352
3094899160,16cyclictest0-21swapper/209:21:502
3094899160,15cyclictest0-21swapper/212:28:402
30946991616,0cyclictest0-21swapper/107:10:121
3094699160,2cyclictest0-21swapper/109:55:471
3094699160,1cyclictest0-21swapper/109:34:511
3094699160,16cyclictest0-21swapper/111:24:071
3094699160,13cyclictest0-21swapper/111:43:571
3094199164,4cyclictest0-21swapper/011:00:150
30941991611,3cyclictest2248-21ls09:55:000
3094899150,14cyclictest0-21swapper/211:55:172
3094899150,14cyclictest0-21swapper/211:55:162
3094699150,1cyclictest0-21swapper/112:26:531
3094699150,1cyclictest0-21swapper/111:10:161
3094699150,14cyclictest0-21swapper/110:57:231
3094699150,13cyclictest0-21swapper/111:51:041
3094199159,3cyclictest13-21ksoftirqd/010:45:220
3094199152,4cyclictest0-21swapper/012:10:180
3095199147,4cyclictest23520-21diskmemload09:59:063
3095199146,6cyclictest0-21swapper/307:19:593
30951991410,3cyclictest3862-21cron10:15:003
30951991410,3cyclictest15850-21tail08:50:273
3095199140,14cyclictest0-21swapper/312:15:123
3095199140,13cyclictest7098-21grep12:30:213
3095199140,13cyclictest0-21swapper/311:57:303
3095199140,13cyclictest0-21swapper/311:57:293
3094899146,5cyclictest1235-21ntpd12:32:222
30948991413,1cyclictest0-21swapper/212:20:132
30948991413,1cyclictest0-21swapper/212:13:432
30948991413,1cyclictest0-21swapper/211:22:132
30948991413,1cyclictest0-21swapper/210:08:542
30948991413,1cyclictest0-21swapper/209:55:142
30948991413,1cyclictest0-21swapper/209:30:242
3094899141,13cyclictest0-21swapper/208:03:152
30948991410,3cyclictest27589-21tail09:50:162
3094899140,14cyclictest0-21swapper/211:11:402
3094899140,12cyclictest20933-21grep07:55:172
3094899140,0cyclictest0-21swapper/210:52:022
30946991413,1cyclictest0-21swapper/111:46:301
30946991413,1cyclictest0-21swapper/110:03:131
30946991411,2cyclictest16470-21idleruntime-cro11:20:001
3094699140,1cyclictest0-21swapper/112:05:131
3094699140,14cyclictest0-21swapper/112:34:141
3094699140,13cyclictest0-21swapper/107:54:171
3094699140,0cyclictest0-21swapper/110:32:071
3094199149,3cyclictest2630-21cat09:35:190
3094199147,5cyclictest30666-21grep10:30:160
3094199145,8cyclictest18784-21tr09:00:000
3094199142,11cyclictest9920-21/usr/sbin/munin11:35:220
30941991411,2cyclictest29627-21grep12:05:200
3094199140,1cyclictest141rcu_preempt12:35:120
3095199139,3cyclictest30579-21sed12:25:183
3095199139,3cyclictest20954-21head07:55:173
30951991310,2cyclictest9356-21tail11:15:163
30951991310,2cyclictest4617-21wc08:30:003
30951991310,2cyclictest26659-21wc11:25:203
30951991310,2cyclictest16093-21sort07:45:173
30951991310,2cyclictest10712-21grep09:40:173
3095199130,12cyclictest2328-21grep11:30:163
3095199130,10cyclictest0-21swapper/310:25:143
3094899139,3cyclictest18465-21grep09:45:132
3094899137,3cyclictest225-21systemd-journal12:05:052
30948991312,1cyclictest0-21swapper/212:04:332
30948991312,1cyclictest0-21swapper/212:04:332
30948991312,1cyclictest0-21swapper/211:15:042
30948991312,1cyclictest0-21swapper/210:45:432
30948991312,1cyclictest0-21swapper/210:02:232
30948991310,2cyclictest27668-21grep11:45:162
3094699139,3cyclictest4970-21grep08:30:141
3094699139,2cyclictest8023-21sleep07:30:001
3094699139,1cyclictest0-21swapper/111:55:231
3094699139,1cyclictest0-21swapper/111:55:221
30946991310,2cyclictest2825-21grep09:35:211
30946991310,2cyclictest27837-21grep08:10:151
30946991310,2cyclictest19148-21head09:45:161
30946991310,2cyclictest15805-21wc12:35:241
3094699130,1cyclictest0-21swapper/111:32:591
3094699130,0cyclictest0-21swapper/109:22:381
3094199139,3cyclictest31994-21turbostat11:10:000
3094199139,3cyclictest26887-21sed11:25:210
3094199135,4cyclictest0-21swapper/010:00:130
3094199132,1cyclictest141rcu_preempt11:12:260
3094199130,12cyclictest9932-21grep07:30:260
3094199130,0cyclictest0-21swapper/010:54:520
3095199129,2cyclictest31195-21grep08:15:213
3095199129,2cyclictest30702-21grep10:30:163
3095199129,2cyclictest25564-21grep09:30:143
3095199129,2cyclictest24841-21head09:10:163
3095199129,2cyclictest23671-21sh11:05:003
3095199129,2cyclictest23594-21cut08:00:163
3095199129,2cyclictest22126-21grep12:20:153
3095199129,2cyclictest20288-21grep09:45:253
3095199129,2cyclictest19218-21grep11:40:153
3095199129,2cyclictest18223-21cat11:20:193
3095199129,2cyclictest1743-21tr09:35:143
3095199129,2cyclictest13685-21sort10:20:173
3095199129,2cyclictest12501-21cut08:45:163
3095199128,3cyclictest9885-21sed07:30:273
3095199128,3cyclictest32137-21ls07:10:203
3095199128,3cyclictest13115-21grep07:40:143
3095199120,2cyclictest0-21swapper/309:20:223
3095199120,0cyclictest0-21swapper/311:35:023
3094899129,2cyclictest31109-21tr08:15:202
3094899129,2cyclictest15300-21wc07:45:002
3094899129,2cyclictest13359-21cat10:20:152
3094899128,3cyclictest48392sleep210:34:352
3094899128,2cyclictest12753-21cron07:40:002
3094899121,2cyclictest537-21snmpd07:49:142
3094899120,11cyclictest683-21sed08:20:162
3094699129,2cyclictest4586-21cat07:20:191
3094699129,2cyclictest306-21tr09:15:151
3094699129,2cyclictest21825-21grep07:55:261
3094699129,2cyclictest17301-21head08:55:161
3094699129,2cyclictest10555-21grep09:40:161
3094699128,4cyclictest17952-21apache210:50:201
3094699128,3cyclictest31256-21ssh09:14:271
3094699128,2cyclictest23074-21tr08:00:141
3094699128,1cyclictest27129-21taskset08:07:431
3094699121,3cyclictest38150irq/131-snd_hda10:25:051
30946991210,1cyclictest8418-21tr07:30:141
30946991210,1cyclictest23520-21diskmemload12:10:281
30946991210,1cyclictest23011-21ssh10:45:141
3094699120,1cyclictest0-21swapper/109:29:361
3094199129,2cyclictest8136-21cut10:55:170
3094199129,2cyclictest6811-21perf08:35:000
3094199129,2cyclictest30971-21grep12:25:200
3094199129,2cyclictest20700-21head10:05:160
3094199129,2cyclictest19840-21wc09:45:220
3094199129,2cyclictest15819-21ssh09:43:300
3094199129,2cyclictest13615-21cron12:35:010
3094199128,3cyclictest7615-21grep08:35:170
3094199128,3cyclictest7615-21grep08:35:160
3094199128,2cyclictest14301-21wc07:40:220
3094199127,3cyclictest32588-21grep09:15:130
3094199127,2cyclictest25649-21rm10:27:070
3094199123,3cyclictest0-21swapper/008:20:130
3094199122,9cyclictest23520-21diskmemload11:50:060
3094199122,5cyclictest2348-21idleruntime-cro11:50:000
3094199120,7cyclictest31981-21awk09:14:590
3094199120,0cyclictest141rcu_preempt11:20:450
3095199119,1cyclictest50482sleep311:50:343
3095199118,2cyclictest951-21ssh10:50:313
3095199118,2cyclictest8731-21sed08:35:273
3095199118,2cyclictest8731-21sed08:35:263
3095199118,2cyclictest8073-21rm10:35:343
3095199118,2cyclictest7952-21taskset10:55:163
3095199118,2cyclictest7293-21ssh12:11:083
3095199118,2cyclictest711-21ssh09:15:173
3095199118,2cyclictest5360-21tune2fs08:30:173
3095199118,2cyclictest5164-21tail10:15:163
3095199118,2cyclictest30493-21ssh11:46:423
3095199118,2cyclictest28643-21wc08:10:193
3095199118,2cyclictest23519-21munin-plugin-st09:10:003
3095199118,2cyclictest23477-21ssh10:45:163
3095199118,2cyclictest23236-21ssh10:06:283
3095199118,2cyclictest18521-21cut07:50:143
3095199118,2cyclictest1722-21taskset08:20:523
3095199118,2cyclictest11772-21taskset07:35:213
3095199118,2cyclictest11765-21ssh10:00:143
3095199117,2cyclictest5975-21tr07:25:133
3095199110,10cyclictest22929-21ssh09:28:303
3094899118,2cyclictest8592-21ssh09:39:262
3094899118,2cyclictest6536-21chrt08:31:462
3094899118,2cyclictest20729-21head09:00:282
3094899118,2cyclictest19950-21cron07:54:592
3094899116,5cyclictest0-21swapper/208:58:532
3094899114,4cyclictest455-21dbus-daemon07:19:592
30948991110,1cyclictest0-21swapper/207:32:232
3094699118,2cyclictest5841-21ssh10:15:211
3094699118,2cyclictest3944-21grep08:25:241
3094699118,2cyclictest28580-21cat10:10:141
3094699118,2cyclictest26230-21ssh12:03:451
3094699118,2cyclictest26230-21ssh12:03:441
3094699118,2cyclictest21002-21cat10:05:201
3094699118,2cyclictest15624-21ssh10:40:221
3094699118,2cyclictest15011-21sh12:15:361
3094699118,2cyclictest13948-21cat10:20:181
3094699118,2cyclictest11588-21basename08:45:011
3094699118,2cyclictest11280-21sort07:35:191
3094699117,3cyclictest8739-21sed08:35:271
3094699117,3cyclictest8739-21sed08:35:261
3094699117,3cyclictest30219-21grep08:15:131
3094699117,3cyclictest15867-21grep08:50:271
3094699113,3cyclictest225-21systemd-journal07:18:031
3094699110,10cyclictest1824-21chrt08:22:111
3094199119,1cyclictest10796-21cat10:00:010
3094199118,2cyclictest7960-21taskset07:29:330
3094199118,2cyclictest5817-21ssh10:35:090
3094199118,2cyclictest4402-21if_enp2s007:20:180
3094199118,2cyclictest4117-21ssh10:15:030
3094199118,2cyclictest3522-21cut08:25:200
3094199118,2cyclictest32078-21cat07:10:180
3094199118,2cyclictest29977-21ssh09:32:250
3094199118,2cyclictest28939-21df10:10:150
3094199118,2cyclictest2733-21cat11:30:190
3094199118,2cyclictest23361-21ssh12:20:290
3094199118,2cyclictest22547-21cat08:00:000
3094199118,2cyclictest19591-21ssh10:42:540
3094199118,2cyclictest19523-21df09:00:160
3094199118,2cyclictest17065-21grep07:45:260
3094199118,2cyclictest14852-21ssh12:15:280
3094199118,2cyclictest11990-21fschecks_count11:55:160
3094199118,2cyclictest11990-21fschecks_count11:55:160
3094199118,2cyclictest10974-21ssh09:20:470
3094199117,3cyclictest32075-21cron08:20:000
3094199117,3cyclictest17644-21sort09:25:160
3095199108,1cyclictest4221-21grep07:20:173
3095199108,1cyclictest18386-21taskset08:55:313
3095199108,1cyclictest114162sleep308:43:123
3095199102,7cyclictest25217-21apt-get08:05:133
3094899108,1cyclictest253232sleep208:05:132
3094899108,1cyclictest11491-21taskset08:44:112
3094899107,2cyclictest2639-21expr08:25:132
3094899107,2cyclictest12549-21head08:45:172
3094899100,9cyclictest5981-21aten2_r7power_p07:25:132
3094699108,1cyclictest19331-21expr09:00:151
3094699107,2cyclictest15771-21expr07:45:151
3094699107,2cyclictest13625-21dnf07:40:171
3094699107,2cyclictest11870-21grep08:45:131
3094199108,1cyclictest23761-21cut08:00:190
3094199108,1cyclictest19781-21ssh12:00:050
3094199108,1cyclictest19781-21ssh12:00:040
3094199107,2cyclictest27550-21taskset08:10:080
3094199107,2cyclictest27219-21chrt08:08:510
3094199107,2cyclictest18335-21df07:50:120
3094199107,2cyclictest15399-21grep08:50:210
3094199107,2cyclictest12355-21head07:35:290
3094199107,2cyclictest1137-21cut07:15:140
3094199105,2cyclictest225-21systemd-journal08:45:010
309489997,1cyclictest31319-21egrep07:10:152
309489997,1cyclictest23325-21chrt09:07:442
309489997,1cyclictest14187-21basename08:50:122
309489996,2cyclictest3535-21apt-get07:20:142
309469996,2cyclictest21807-21cat09:05:161
309419997,1cyclictest137242sleep008:47:220
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional