You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-27 - 08:25

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot4s.osadl.org (updated Wed May 27, 2026 00:45:10)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
7391994822,18cyclictest0-21swapper/321:45:123
739199451,27cyclictest35966irq/58-eth023:15:133
735599441,33cyclictest11848-21ssh21:20:130
739199423,25cyclictest35966irq/58-eth021:25:133
739199423,24cyclictest35966irq/58-eth022:15:123
735599421,2cyclictest131rcu_preempt21:35:140
739199416,21cyclictest35966irq/58-eth000:30:143
7391994111,17cyclictest35966irq/58-eth023:10:153
735599401,28cyclictest32410-21ssh00:30:140
739199386,15cyclictest0-21swapper/322:50:143
7391993820,11cyclictest35966irq/58-eth021:15:143
7391993622,5cyclictest0-21swapper/320:45:123
739199353,20cyclictest35966irq/58-eth022:35:113
739199342,22cyclictest35966irq/58-eth000:05:153
739199342,22cyclictest35966irq/58-eth000:05:153
738099347,12cyclictest27-21ksoftirqd/221:45:302
738099342,16cyclictest10005-21usage_EthcTask22:10:302
739199338,16cyclictest321rcuc/323:55:133
7391993321,8cyclictest0-21swapper/300:15:133
735599331,27cyclictest25632-21diskmemload00:05:160
735599331,27cyclictest25632-21diskmemload00:05:160
7391993216,11cyclictest321rcuc/322:40:123
739199313,19cyclictest35966irq/58-eth023:35:133
7391993016,8cyclictest35966irq/58-eth023:45:253
739199301,20cyclictest35966irq/58-eth023:25:133
7380993017,8cyclictest27-21ksoftirqd/221:44:312
735599303,3cyclictest131rcu_preempt22:55:140
735599301,22cyclictest8472-21usage_Anybus22:45:300
738099291,16cyclictest9013-21ssh00:35:152
7369992922,3cyclictest21-21ksoftirqd/119:55:111
739199283,16cyclictest0-21swapper/322:10:303
739199282,16cyclictest35966irq/58-eth000:35:143
738099282,22cyclictest0-21swapper/222:17:132
738099282,21cyclictest30782-21usage_EthcTask23:15:302
738099282,19cyclictest0-21swapper/219:50:132
7380992815,10cyclictest0-21swapper/223:40:182
7380992811,6cyclictest0-21swapper/223:05:012
738099281,14cyclictest25632-21diskmemload22:50:152
738099281,11cyclictest11421-21ssh22:48:302
736999282,18cyclictest17114-21ssh23:10:141
7369992819,6cyclictest21-21ksoftirqd/121:55:131
735599282,20cyclictest27247-21ssh23:32:210
735599282,19cyclictest1466-21/usr/sbin/munin21:50:300
735599281,20cyclictest7401-21cat23:20:300
735599281,20cyclictest20102-21ssh22:16:300
735599281,19cyclictest10086-21cat22:10:300
739199273,16cyclictest0-21swapper/322:45:303
7391992710,12cyclictest0-21swapper/300:10:143
738099273,7cyclictest0-21swapper/223:34:302
738099273,21cyclictest16000-21munin-plugin-st21:00:012
738099273,17cyclictest13890-21kthreadcore21:55:222
738099272,21cyclictest22978-21ssh23:48:302
738099272,11cyclictest0-21swapper/220:45:142
7380992714,11cyclictest0-21swapper/200:25:132
738099271,17cyclictest26282-21switchtime22:55:292
739199263,16cyclictest0-21swapper/322:25:303
739199261,17cyclictest0-21swapper/323:00:113
738099263,7cyclictest0-21swapper/223:20:302
738099262,9cyclictest24468-21ssh23:13:312
738099262,21cyclictest15368-21ssh21:20:232
739199253,14cyclictest0-21swapper/323:20:303
739199252,12cyclictest0-21swapper/323:30:123
739199251,14cyclictest35966irq/58-eth021:35:143
738099253,19cyclictest4144-21fschecks_time19:40:152
738099252,8cyclictest17446-21ssh23:26:302
738099252,18cyclictest3225-21ssh22:44:302
738099251,21cyclictest2380-21kthreadcore23:55:192
738099251,21cyclictest15737-21kthreadcore20:25:202
738099251,19cyclictest5257-21cpuspeed20:15:142
738099251,14cyclictest1285066ssh22:30:232
738099242,6cyclictest0-21swapper/223:39:302
738099242,17cyclictest0-21swapper/222:07:112
738099242,14cyclictest16431-21fschecks_time21:00:172
738099242,13cyclictest21092-21fschecks_count21:25:162
739199233,13cyclictest0-21swapper/322:34:303
739199232,15cyclictest321rcuc/321:40:113
739199231,14cyclictest35966irq/58-eth021:00:133
738099233,13cyclictest20760-21cut00:05:232
738099233,13cyclictest20760-21cut00:05:232
738099232,16cyclictest0-21swapper/222:39:302
738099232,16cyclictest0-21swapper/222:22:042
738099232,16cyclictest0-21swapper/222:01:302
738099232,16cyclictest0-21swapper/221:54:302
738099231,8cyclictest0-21swapper/221:38:452
7380992312,9cyclictest13052-21ssh23:05:392
7380992312,8cyclictest0-21swapper/219:35:262
7380992311,5cyclictest31177-21ssh23:52:332
7369992316,4cyclictest23532-21sh22:20:151
7369992312,8cyclictest16867-21cron19:20:011
7369992312,8cyclictest16867-21cron19:20:011
735599231,3cyclictest131rcu_preempt20:30:130
739199223,13cyclictest0-21swapper/321:52:303
7391992210,8cyclictest0-21swapper/321:30:143
7391992210,8cyclictest0-21swapper/321:30:143
738099222,14cyclictest0-21swapper/222:26:462
738099222,11cyclictest0-21swapper/220:30:142
736999224,4cyclictest131rcu_preempt21:35:151
736999222,17cyclictest23333-21ssh00:25:111
7369992217,2cyclictest0-21swapper/121:45:111
7369992214,3cyclictest21-21ksoftirqd/123:20:121
7369992212,6cyclictest11864-21ssh21:20:141
736999221,18cyclictest27398-21ssh22:40:131
736999221,17cyclictest15737-21ssh21:40:121
736999220,3cyclictest22314-21ssh22:55:141
735599229,9cyclictest26133-21df21:10:150
735599222,14cyclictest30769-21ssh23:15:300
735599222,14cyclictest1-21systemd21:47:000
735599221,6cyclictest709-21rs:main0
735599221,6cyclictest16922-21latency_hist19:20:010
735599221,6cyclictest16922-21latency_hist19:20:010
739199214,11cyclictest0-21swapper/323:05:143
739199214,10cyclictest35966irq/58-eth000:25:113
739199211,15cyclictest0-21swapper/321:55:163
738099213,16cyclictest7138-21kthreadcore00:15:212
738099213,11cyclictest6503-21/usr/sbin/munin21:15:282
738099212,17cyclictest14418-21fschecks_count00:20:162
738099212,16cyclictest897-21kthreadcore21:30:232
738099212,16cyclictest897-21kthreadcore21:30:232
738099212,16cyclictest14736-21grep20:25:012
7380992112,6cyclictest0-21swapper/200:30:112
738099211,13cyclictest27346irq/40-dwc2_hso20:40:132
736999212,4cyclictest131rcu_preempt22:15:111
7369992114,4cyclictest23602-21threads22:00:311
736999211,3cyclictest131rcu_preempt00:05:161
736999211,3cyclictest131rcu_preempt00:05:161
7369992112,7cyclictest6367-21cron20:50:011
7369992111,7cyclictest13360-21cron19:50:001
735599212,16cyclictest16526-21kthreadcore20:25:200
735599212,14cyclictest2388-21ssh22:43:300
735599211,15cyclictest22179-21kworker/u8:222:25:140
7355992110,8cyclictest16370-21kthreadcore23:45:180
738099202,16cyclictest19176-21kthreadcore19:55:182
738099202,15cyclictest26628-21irqrtprio21:10:192
738099202,11cyclictest27449-21kthreadcore20:35:232
738099202,11cyclictest17171-21cpuspeed19:20:142
738099201,17cyclictest11178-21cron20:55:012
738099201,16cyclictest29006-21irqcore00:10:192
738099201,16cyclictest16871-21cron19:20:012
738099201,16cyclictest16871-21cron19:20:012
738099201,16cyclictest13442-21latency_hist19:50:022
738099201,15cyclictest10478-21df00:00:162
738099201,12cyclictest25031-21kthreadcore19:25:212
736999209,8cyclictest21664-21cron19:25:011
736999206,4cyclictest131rcu_preempt22:25:221
736999206,3cyclictest131rcu_preempt00:20:021
736999204,13cyclictest5328-21ssh22:10:131
736999203,14cyclictest1212-21ssh23:55:131
736999202,14cyclictest24650-21ssh23:50:141
7369992014,4cyclictest7341-21cyclictest21:10:141
736999201,14cyclictest31508-21ssh23:00:131
7369992010,7cyclictest16745-21cut23:45:171
7369992010,7cyclictest15984-21cron21:00:011
735599203,15cyclictest11389-21kthreadcore20:20:210
735599203,14cyclictest27944-21tr23:50:220
735599202,15cyclictest9053-21unixbench_singl20:15:280
735599202,15cyclictest7695-21apt23:05:000
735599202,15cyclictest31347-21kthreadcore21:30:210
735599202,15cyclictest31347-21kthreadcore21:30:210
735599202,15cyclictest31008-21kthreadcore20:40:200
735599201,3cyclictest0-21swapper/000:00:160
7355992012,6cyclictest0-21swapper/019:55:250
735599201,16cyclictest23822-21kthreadcore21:25:220
735599201,14cyclictest27222-21ssh22:20:270
735599201,13cyclictest17133-21ssh23:10:140
739199198,7cyclictest35966irq/58-eth022:20:163
739199197,9cyclictest0-21swapper/322:58:013
739199193,11cyclictest0-21swapper/323:53:303
739199193,11cyclictest0-21swapper/322:04:313
739199192,12cyclictest0-21swapper/300:20:123
739199192,11cyclictest0-21swapper/319:50:143
739199192,11cyclictest0-21swapper/319:40:133
7391991912,5cyclictest0-21swapper/323:40:243
7391991911,5cyclictest35966irq/58-eth022:05:123
739199191,13cyclictest0-21swapper/321:10:303
739199191,13cyclictest0-21swapper/319:15:273
739199191,13cyclictest0-21swapper/319:15:273
739199191,12cyclictest0-21swapper/320:50:293
738099198,9cyclictest26881-21sshd20:00:282
738099191,8cyclictest30651-21kthreadcore20:05:222
738099191,16cyclictest3680-21latency20:10:232
738099191,10cyclictest7649-21fschecks_time19:10:162
738099191,10cyclictest26739-21cpu19:30:132
736999199,7cyclictest31567-21apt-get19:35:121
736999199,7cyclictest26979-21cron22:40:011
736999199,7cyclictest20660-21cut23:30:021
736999195,11cyclictest167-21systemd-network23:38:381
736999194,3cyclictest131rcu_preempt00:30:471
736999192,14cyclictest26107-21ssh23:15:121
736999192,13cyclictest9893-21apt-get00:00:031
7369991912,4cyclictest0-21swapper/119:10:261
7369991911,6cyclictest12174-21/usr/sbin/munin23:05:301
736999191,15cyclictest13597-21apt-get19:50:131
736999191,14cyclictest2696-21ssh21:15:141
7369991910,6cyclictest7435-21tr23:40:181
735599198,3cyclictest12-21ksoftirqd/022:04:460
735599193,13cyclictest13263-21kthreadcore20:55:220
735599192,15cyclictest2361-21irqrtprio20:45:180
735599192,15cyclictest20029-21timerwakeupswit21:00:290
735599192,14cyclictest2641-21/usr/sbin/munin21:15:160
735599192,14cyclictest24494-21tr00:25:170
735599192,12cyclictest5718-21ssh23:39:300
735599191,9cyclictest10452-21ssh00:35:210
735599191,16cyclictest29705-21kthreadcore19:30:210
735599191,15cyclictest27922-21netstat20:35:230
735599191,15cyclictest26014-21kthreadcore20:00:210
735599191,15cyclictest2403-21latency19:35:210
735599191,15cyclictest12069-21timerandwakeup23:05:290
7355991910,7cyclictest1308-21df_inode23:55:140
7355991910,5cyclictest0-21swapper/023:25:140
735599190,15cyclictest13165-21sh22:50:150
739199189,6cyclictest0-21swapper/300:04:363
739199182,10cyclictest0-21swapper/320:55:143
739199182,10cyclictest0-21swapper/319:30:133
739199181,13cyclictest0-21swapper/320:35:173
738099181,15cyclictest21819-21cut21:05:202
736999189,6cyclictest26087-21sshd19:25:291
736999188,7cyclictest9506-21cut00:35:191
736999182,4cyclictest131rcu_preempt00:10:131
736999181,9cyclictest16786-21netstat22:50:251
7369991812,4cyclictest0-21swapper/121:05:171
736999181,14cyclictest28697-21sh22:05:121
736999181,14cyclictest20905-21ssh21:25:131
7369991811,4cyclictest0-21swapper/121:50:141
736999181,13cyclictest7459-21ntp_states19:40:251
736999181,13cyclictest11390-21tr22:30:191
736999180,9cyclictest2506-21kthreadcore20:10:211
735599186,3cyclictest25632-21diskmemload00:10:140
735599182,14cyclictest8749-21kthreadcore19:10:190
735599182,14cyclictest10991-21usage_EthcTask20:50:310
735599182,11cyclictest13122-21ssh23:42:300
7355991812,4cyclictest0-21swapper/019:50:290
7355991812,4cyclictest0-21swapper/019:25:280
735599181,15cyclictest6446-21kthreadcore00:15:200
735599181,15cyclictest4886-21kthreadcore19:40:190
735599181,14cyclictest11317-21kthreadcore21:55:180
739199179,5cyclictest35966irq/58-eth021:20:143
739199173,9cyclictest0-21swapper/320:25:233
739199172,9cyclictest0-21swapper/320:05:133
739199172,10cyclictest0-21swapper/320:20:153
739199171,10cyclictest0-21swapper/320:40:163
736999178,6cyclictest21-21ksoftirqd/121:00:141
736999177,8cyclictest24277-21cat23:30:211
736999176,9cyclictest12169-21cut20:20:221
736999174,9cyclictest21-21ksoftirqd/120:25:141
7369991711,4cyclictest0-21swapper/120:05:011
736999171,13cyclictest30128-21df_inode21:30:161
736999171,13cyclictest30128-21df_inode21:30:161
7369991711,3cyclictest29257-21cat20:40:011
736999171,12cyclictest32580-21cron20:10:011
736999171,12cyclictest29480-21grep20:40:161
7369991710,3cyclictest131rcu_preempt00:20:201
736999170,9cyclictest24209-21latency_hist20:35:021
736999170,9cyclictest0-21swapper/122:45:171
735599179,6cyclictest14331-21cat00:20:140
735599178,7cyclictest9000-21fw_packets19:45:140
735599178,5cyclictest12-21ksoftirqd/020:05:130
735599171,8cyclictest18656-21ssh22:35:160
735599171,6cyclictest10020-21cut22:30:180
735599171,4cyclictest28906-21ssh22:05:150
735599171,13cyclictest3909-21proc_pri20:10:270
739199162,9cyclictest0-21swapper/320:00:143
739199162,9cyclictest0-21swapper/319:20:143
739199162,10cyclictest0-21swapper/320:10:143
739199162,10cyclictest0-21swapper/319:55:123
739199161,10cyclictest0-21swapper/321:05:233
738099161,6cyclictest7459-21latency_hist19:10:022
736999163,10cyclictest7662-21kthreadcore20:50:201
736999161,12cyclictest5335-21fschecks_count20:15:161
736999161,12cyclictest30236-21mailstats19:30:261
735599165,8cyclictest18424-21kthreadcore19:20:200
735599162,12cyclictest23231-21kthreadcore21:05:220
739199152,9cyclictest0-21swapper/319:35:133
739199152,9cyclictest0-21swapper/319:10:143
739199147,5cyclictest0-21swapper/320:30:233
739199141,8cyclictest0-21swapper/319:30:023
739199141,7cyclictest0-21swapper/319:50:023
739199141,6cyclictest0-21swapper/320:15:193
735599140,11cyclictest7366-21ls19:10:020
736999131,6cyclictest7376-21latency_hist19:10:011
739199101,7cyclictest0-21swapper/319:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional