You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-03 - 20:23

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Mon Aug 03, 2026 12:45:18)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1968199591,46cyclictest5931-21ssh09:10:130
1968199565,18cyclictest20686-21ssh12:00:140
1968199501,35cyclictest473-21ssh12:25:120
19711994823,15cyclictest35966irq/58-eth008:10:123
19711994823,15cyclictest35966irq/58-eth008:10:123
1968999462,40cyclictest27763-21ssh11:10:151
1968999452,39cyclictest11504-21ssh10:25:121
1968999436,9cyclictest131rcu_preempt12:20:121
1968999436,9cyclictest131rcu_preempt12:20:111
1968999421,30cyclictest7819-21ssh10:05:111
1969899413,21cyclictest27-21ksoftirqd/210:20:122
1968199402,14cyclictest22113-21cat09:35:230
1971199393,17cyclictest35966irq/58-eth011:10:143
1968999390,33cyclictest9540-21ssh12:30:131
1968999390,33cyclictest9540-21ssh12:30:131
1968199394,26cyclictest13253-21ssh11:20:120
1971199381,23cyclictest35966irq/58-eth011:00:153
19698993825,8cyclictest27-21ksoftirqd/209:20:122
1968199372,14cyclictest16624-21ssh11:03:000
1971199362,21cyclictest35966irq/58-eth012:20:133
1971199362,21cyclictest35966irq/58-eth012:20:133
19711993515,14cyclictest35966irq/58-eth011:05:143
1968199352,26cyclictest9599-21ssh09:30:130
1968199352,26cyclictest9599-21ssh09:30:120
1968199352,10cyclictest0-21swapper/011:25:140
1968199351,26cyclictest31283-21ssh11:30:110
1968199351,26cyclictest31283-21ssh11:30:100
1969899343,20cyclictest27346irq/40-dwc2_hso08:10:132
1969899343,20cyclictest27346irq/40-dwc2_hso08:10:132
1968999342,25cyclictest29495-21ssh10:35:161
19698993324,5cyclictest27-21ksoftirqd/208:25:122
1968999332,22cyclictest9742-21ssh11:00:151
1969899322,16cyclictest27655-21ssh09:40:132
1968199328,8cyclictest14202-21df_inode08:45:170
1971199314,22cyclictest0-21swapper/310:05:123
19711993118,8cyclictest35966irq/58-eth010:25:123
19698993113,13cyclictest0-21swapper/212:05:132
1969899311,10cyclictest19671-21cyclictest10:00:142
19698993022,3cyclictest0-21swapper/211:15:132
19689993015,3cyclictest21-21ksoftirqd/111:35:121
1968199302,24cyclictest7864-21ssh11:35:140
19711992921,5cyclictest35966irq/58-eth011:20:133
1969899296,15cyclictest27346irq/40-dwc2_hso11:10:132
1969899293,13cyclictest15097-21ssh12:15:152
1969899292,21cyclictest8425-21ssh10:59:002
1968999294,20cyclictest2477-21ssh10:20:131
1968999291,23cyclictest0-21swapper/108:10:121
1968999291,23cyclictest0-21swapper/108:10:111
1968199292,21cyclictest5670-21ssh12:09:590
1969899282,19cyclictest27346irq/40-dwc2_hso08:50:122
1969899282,19cyclictest12699-21ssh09:30:222
1969899282,19cyclictest12699-21ssh09:30:222
1968999282,23cyclictest31227-21ssh10:00:121
1968999281,13cyclictest21-21ksoftirqd/107:20:141
1968999281,13cyclictest21-21ksoftirqd/107:20:141
1968199282,19cyclictest844-21munin-node07:50:150
1968199281,21cyclictest29510-21ssh10:35:160
19681992810,12cyclictest6105-21sh12:10:140
19698992713,11cyclictest0-21swapper/208:35:172
1969899271,23cyclictest28347-21basename09:00:012
19681992716,6cyclictest16843-21ssh10:10:130
1971199266,11cyclictest0-21swapper/310:35:163
1971199262,17cyclictest35966irq/58-eth009:40:133
1971199262,14cyclictest35966irq/58-eth009:25:113
19711992615,7cyclictest35966irq/58-eth010:50:143
1969899262,8cyclictest29148-21ssh09:58:002
1969899262,8cyclictest29148-21ssh09:58:002
1969899262,20cyclictest4661-21kthreadcore11:50:212
1969899261,6cyclictest0-21swapper/210:05:122
1969899261,17cyclictest4294-21ls09:45:142
1969899261,16cyclictest26774-21kthreadcore07:15:222
19689992616,6cyclictest0-21swapper/109:45:131
1968199262,18cyclictest15035-21ssh10:45:120
1971199258,11cyclictest35966irq/58-eth010:20:153
19711992513,8cyclictest35966irq/58-eth010:15:253
1969899255,13cyclictest1987-21munin-run10:20:012
1969899252,8cyclictest15618-21diskmemload10:25:292
1969899251,16cyclictest21580-21kthreadcore07:10:192
19698992511,11cyclictest16028-21ssh11:55:312
1968999252,19cyclictest201rcuc/112:15:141
1968999251,6cyclictest9699-21df09:30:151
1968999251,6cyclictest9699-21df09:30:151
1968999251,18cyclictest6080-21ssh10:40:131
1968199254,17cyclictest4274-21ssh09:25:250
1968199252,19cyclictest4236-21ssh09:45:140
1971199243,14cyclictest35966irq/58-eth008:20:123
19698992413,9cyclictest13235-21ssh09:13:292
1969899241,12cyclictest523-21ssh09:25:132
19698992410,6cyclictest16766systemd-network07:23:082
19698992410,6cyclictest16766systemd-network07:23:082
1968999245,3cyclictest131rcu_preempt10:15:251
1968999241,18cyclictest18741-21ssh11:05:151
1968999241,17cyclictest16862-21ssh10:10:141
1968999241,16cyclictest11207-21apt-get11:55:011
1968199244,13cyclictest26059-21ssh11:45:150
1968199241,19cyclictest15054-21ssh12:15:120
1971199233,16cyclictest0-21swapper/312:35:133
19711992311,9cyclictest35966irq/58-eth009:10:143
1969899232,7cyclictest0-21swapper/212:20:132
1969899232,7cyclictest0-21swapper/212:20:122
1969899232,6cyclictest0-21swapper/209:35:592
1969899232,17cyclictest0-21swapper/207:30:152
1969899232,17cyclictest0-21swapper/207:30:142
1969899232,16cyclictest0-21swapper/212:15:002
1969899232,16cyclictest0-21swapper/209:50:592
1969899232,14cyclictest0-21swapper/212:30:132
1969899232,14cyclictest0-21swapper/212:30:122
19698992311,9cyclictest3103-21ssh10:36:582
1969899231,13cyclictest313-21latency_hist10:55:012
19698992310,10cyclictest20612-21df_inode10:30:162
1968999232,16cyclictest844-21munin-node09:25:121
1968999231,7cyclictest17600-21cat11:20:301
1968999231,18cyclictest19833-21apt-get07:10:121
1968199232,15cyclictest20516-21ssh10:30:150
1968199231,16cyclictest24070-21ssh10:50:140
1971199225,13cyclictest35966irq/58-eth011:55:133
1971199223,13cyclictest0-21swapper/310:10:143
1971199223,12cyclictest35966irq/58-eth009:45:123
1971199223,12cyclictest0-21swapper/310:40:143
1971199223,12cyclictest0-21swapper/309:35:133
1971199221,17cyclictest35966irq/58-eth007:55:153
1969899224,15cyclictest20357-21kthreadcore08:15:222
1969899222,6cyclictest0-21swapper/212:25:412
1969899221,18cyclictest18380-21ssh12:35:052
19698992211,3cyclictest7267-21kthreadcore10:40:202
1969899221,13cyclictest11249-21kthreadcore11:35:232
1969899221,12cyclictest9741-21sh11:00:142
1968999224,14cyclictest25213-21tr10:50:201
1968999222,7cyclictest6043-21diskstats09:10:161
19689992214,6cyclictest892-21ssh10:55:181
1968199228,3cyclictest131rcu_preempt10:05:290
1968199224,10cyclictest0-21swapper/007:10:130
1968199222,14cyclictest32319-21ssh09:40:300
19681992212,3cyclictest5531-21diskmemload10:24:440
1971199219,7cyclictest0-21swapper/310:00:123
1971199217,10cyclictest35966irq/58-eth011:25:143
1971199214,13cyclictest0-21swapper/312:10:133
1971199214,13cyclictest0-21swapper/310:30:273
1971199212,13cyclictest0-21swapper/311:35:143
1969899214,10cyclictest18128-21users11:20:322
1969899213,9cyclictest16350-21kthreadcore08:45:232
1969899212,3cyclictest131rcu_preempt11:08:092
1969899211,7cyclictest0-21swapper/211:34:522
1969899211,7cyclictest0-21swapper/211:34:522
19698992115,4cyclictest21876-21ssh12:00:212
1969899211,18cyclictest18319-21cut11:40:222
1969899211,17cyclictest16852-21ssh10:10:132
1969899211,11cyclictest25701-21munin-plugin-st07:50:012
1969899211,11cyclictest19998-21timerandwakeup07:40:262
19698992110,8cyclictest2203266ssh09:18:112
1968999212,14cyclictest18544-21ssh09:15:231
19689992114,5cyclictest26060-21ssh11:45:151
19689992114,5cyclictest0-21swapper/112:35:121
19689992113,6cyclictest24960-21ssh12:00:321
1968199219,3cyclictest131rcu_preempt10:00:010
1968199219,3cyclictest131rcu_preempt10:00:000
1968199211,16cyclictest688-21ssh10:55:160
1968199211,13cyclictest22491-21ls12:35:290
1971199208,6cyclictest0-21swapper/311:30:113
1971199208,6cyclictest0-21swapper/311:30:113
1971199204,12cyclictest35966irq/58-eth010:55:133
1971199203,12cyclictest0-21swapper/312:29:593
1971199203,11cyclictest35966irq/58-eth012:05:123
1971199202,10cyclictest35966irq/58-eth008:35:113
1971199201,15cyclictest35966irq/58-eth012:15:143
1971199201,13cyclictest0-21swapper/311:50:143
1971199201,13cyclictest0-21swapper/307:30:303
1971199201,13cyclictest0-21swapper/307:30:303
1969899203,14cyclictest29129-21kthreadcore11:45:232
1969899203,14cyclictest16082-21latency_hist07:40:002
1969899202,11cyclictest0-21swapper/207:55:042
1969899201,5cyclictest0-21swapper/210:48:402
19698992013,5cyclictest22284-21ssh11:25:132
1969899201,16cyclictest957-21find09:05:142
1969899201,16cyclictest29696-21tail07:50:302
1969899201,12cyclictest25789-21cat08:20:232
1969899201,10cyclictest5106-21kthreadcore07:25:242
19689992013,5cyclictest5318-21ssh12:25:311
19689992013,5cyclictest30040-21ssh12:05:181
19689992012,5cyclictest5531-21diskmemload09:40:131
1968999201,16cyclictest27410-21ls09:40:011
1968999201,15cyclictest14111-21apt-get08:45:151
19689992010,7cyclictest24766-21cat10:30:311
1968999200,17cyclictest7808-21ls08:35:211
1968199204,11cyclictest0-21swapper/010:19:000
1968199203,12cyclictest14903-21ssh09:15:130
1968199202,4cyclictest24016-21apt-get12:20:120
1968199202,4cyclictest24016-21apt-get12:20:120
1968199202,14cyclictest11701-21sh11:55:140
1968199202,12cyclictest19319-21ssh09:51:590
1968199201,13cyclictest22561-21ssh11:05:290
19681992010,7cyclictest12794-21apt-get08:10:120
19681992010,7cyclictest12794-21apt-get08:10:110
1971199199,7cyclictest0-21swapper/311:45:303
1971199193,8cyclictest35966irq/58-eth009:30:133
1971199193,8cyclictest35966irq/58-eth009:30:123
1971199193,8cyclictest0-21swapper/307:50:133
1971199192,11cyclictest0-21swapper/311:15:143
19711991911,5cyclictest0-21swapper/312:00:263
1971199191,13cyclictest0-21swapper/309:55:133
1971199191,13cyclictest0-21swapper/309:55:123
1971199191,13cyclictest0-21swapper/309:50:163
1969899191,9cyclictest1157-21kthreadcore08:30:212
1969899191,8cyclictest3910-21kthreadcore08:00:202
1969899191,14cyclictest29141-21irqrtprio09:00:192
1969899191,11cyclictest9966-21tr08:40:192
1968999197,9cyclictest22276-21basename08:20:011
19689991913,3cyclictest18779-21processes10:45:281
19689991912,5cyclictest23987-21ssh09:20:131
1968199199,8cyclictest9308-21apt-get08:40:130
1968199199,7cyclictest14396-21kthreadcore10:25:210
1968199193,12cyclictest23971-21ssh09:20:120
1968199192,15cyclictest1946-21cpuspeed07:25:140
1968199191,8cyclictest11305-21apt-config11:55:000
1968199191,14cyclictest31304-21kthreadcore07:20:210
1968199191,14cyclictest31304-21kthreadcore07:20:200
19681991910,6cyclictest31850-21cron08:30:010
1971199187,7cyclictest0-21swapper/309:20:293
1971199182,9cyclictest0-21swapper/308:45:163
1971199182,10cyclictest0-21swapper/310:45:123
1971199182,10cyclictest0-21swapper/308:15:053
1971199181,13cyclictest0-21swapper/307:40:193
1969899181,9cyclictest19773-21latency_hist07:10:022
1969899181,8cyclictest8461-21sed08:05:192
1968999186,9cyclictest27633-21tail08:25:181
1968999183,8cyclictest0-21swapper/108:55:121
1968999182,12cyclictest6305-21ssh11:50:271
1968999182,11cyclictest6207-21sh12:10:161
1968999181,9cyclictest11458-21/usr/sbin/munin07:35:301
1968999181,8cyclictest22990-21tail07:45:211
1968999181,8cyclictest17341-21grep09:50:291
1968999181,6cyclictest20932-21/usr/sbin/munin11:40:301
19689991811,5cyclictest7903-21ssh11:15:271
19689991811,4cyclictest24423-21kthreadcore09:55:221
19689991811,4cyclictest24423-21kthreadcore09:55:221
19689991810,6cyclictest6968-21ssh11:34:281
19689991810,6cyclictest6968-21ssh11:34:281
1968199188,7cyclictest3848-21cat11:15:020
1968199183,9cyclictest0-21swapper/010:03:580
1968199182,14cyclictest26979-21kthreadcore08:55:220
1968199182,13cyclictest6217-21fschecks_count10:40:160
1968199180,15cyclictest28271-21ls07:15:280
1968199180,15cyclictest11732-21ls08:05:290
1971199179,6cyclictest0-21swapper/311:43:203
1971199178,7cyclictest0-21swapper/309:15:203
1971199172,11cyclictest0-21swapper/307:25:033
1971199172,10cyclictest0-21swapper/312:30:143
1971199172,10cyclictest0-21swapper/312:30:143
1968999178,7cyclictest29025-21kthreadcore07:50:241
1968999178,6cyclictest392-21kthreadcore07:55:231
1968999171,9cyclictest8696-21kthreadcore08:05:201
19689991711,4cyclictest0-21swapper/108:50:231
1968999171,13cyclictest7723-21sh08:05:011
1968999171,13cyclictest16505-21fschecks_time07:40:141
1968999171,12cyclictest25991-21ntp_states11:25:271
1968999170,9cyclictest1660-21kthreadcore08:30:211
1968199178,7cyclictest4654-21perl08:35:140
1968199172,11cyclictest9545-21ssh12:30:130
1968199172,11cyclictest9545-21ssh12:30:120
1968199172,10cyclictest0-21swapper/007:35:150
1968199171,13cyclictest16997-21ssh11:40:130
1968199171,12cyclictest30458-21apt-get07:55:030
1968199171,11cyclictest9451-21ssh11:16:000
1968199170,9cyclictest26070-21mailstats08:20:280
1968199170,15cyclictest24872-21ls07:45:290
1971199161,3cyclictest0-21swapper/308:30:143
1968999169,5cyclictest29205-21cron07:20:001
1968999168,6cyclictest12265-21cut08:40:221
1968999161,13cyclictest24994-21cut08:20:211
1968999161,12cyclictest7768-21cut07:30:211
1968999161,12cyclictest7768-21cut07:30:201
1968999161,12cyclictest3831-21cut07:25:221
1968999161,12cyclictest29355-21tr09:00:201
19689991610,4cyclictest0-21swapper/109:05:281
1968199165,9cyclictest22246-21cron08:20:000
1968199164,9cyclictest3299-21tune2fs08:00:160
1968199164,9cyclictest2151-21kthreadcore08:30:220
1968199162,11cyclictest8412-21kthreadcore07:30:220
1968199162,11cyclictest8412-21kthreadcore07:30:220
1968199162,10cyclictest0-21swapper/009:05:150
1968199162,10cyclictest0-21swapper/007:40:120
1968199161,11cyclictest28676-21diskstats09:00:150
1971199152,9cyclictest0-21swapper/308:55:143
1971199151,9cyclictest0-21swapper/308:00:213
1971199151,8cyclictest0-21swapper/307:20:173
1971199151,8cyclictest0-21swapper/307:20:163
1971199151,11cyclictest0-21swapper/308:25:163
1968199151,11cyclictest19267-21irqcore08:50:160
1971199144,5cyclictest0-21swapper/307:45:133
1971199143,8cyclictest0-21swapper/309:00:293
1971199141,7cyclictest0-21swapper/308:45:003
1971199141,4cyclictest0-21swapper/308:50:133
1971199141,10cyclictest0-21swapper/307:15:283
1968999141,10cyclictest19803-21latency_hist07:10:011
1968199142,9cyclictest19592-21latency_hist07:10:020
1971199134,5cyclictest0-21swapper/307:10:143
1971199132,8cyclictest0-21swapper/308:05:163
1971199132,8cyclictest0-21swapper/307:40:013
1971199131,6cyclictest0-21swapper/309:05:253
1971199111,7cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional