You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-30 - 22:33

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Tue Jun 30, 2026 12:45:11)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3251999504,36cyclictest23852-21ssh12:30:131
3253299492,27cyclictest7433-21ssh11:45:132
3251999492,34cyclictest24800-21ssh10:25:141
3251099481,8cyclictest131rcu_preempt11:00:130
3254599475,24cyclictest35966irq/58-eth012:00:133
3251099454,34cyclictest29141-21ssh12:15:110
3251099452,35cyclictest3686-21ssh11:25:120
3254599443,26cyclictest35966irq/58-eth010:20:133
3251099444,37cyclictest10504-21ssh10:35:110
3251099444,33cyclictest20926-21ssh10:05:120
3251099441,29cyclictest21804-21ssh11:35:120
3251999434,28cyclictest30820-21ssh11:40:131
3253299421,20cyclictest19672-21ssh10:40:152
3251999422,31cyclictest31384-21ssh09:35:131
3251099421,28cyclictest24816-21ssh10:25:140
32510994122,10cyclictest131rcu_preempt08:55:150
3253299392,29cyclictest1861-21cut12:00:012
32510993927,8cyclictest12-21ksoftirqd/008:40:120
3251099391,25cyclictest9045-21ssh11:10:140
3251099383,25cyclictest28756-21ssh10:45:140
3254599372,22cyclictest35966irq/58-eth009:55:153
3253299372,19cyclictest25041-21ssh09:49:122
3251099375,19cyclictest479-21ssh12:35:130
3251099373,31cyclictest2102-21ssh12:00:130
3251099371,29cyclictest26130-21ssh09:50:120
3253299362,15cyclictest23848-21ssh12:30:132
3251999361,27cyclictest0-21swapper/108:30:131
3251099369,20cyclictest12-21ksoftirqd/011:50:130
32545993515,14cyclictest35966irq/58-eth010:55:153
3253299351,17cyclictest10332-21ssh12:04:322
3254599343,17cyclictest35966irq/58-eth009:20:123
32545993412,14cyclictest35966irq/58-eth011:20:133
32545993412,14cyclictest35966irq/58-eth011:20:123
3251999341,27cyclictest0-21swapper/109:25:121
3251099346,23cyclictest19800-21kworker/u8:210:10:140
3251099342,24cyclictest14803-21ssh12:25:120
3254599332,21cyclictest35966irq/58-eth009:30:123
3253299331,20cyclictest32446-21ssh11:05:122
3251099333,21cyclictest12-21ksoftirqd/008:25:120
3254599324,21cyclictest0-21swapper/309:45:143
3254599321,16cyclictest0-21swapper/312:05:133
3253299321,9cyclictest0-21swapper/210:18:012
3251099327,21cyclictest30835-21ssh11:40:140
3251099323,19cyclictest5364-21diskmemload10:50:120
3251099322,20cyclictest20204-21ssh12:10:130
3251099316,7cyclictest131rcu_preempt07:53:010
3251099313,16cyclictest1430-21ssh10:30:120
3251099311,23cyclictest12771-21ssh11:30:140
3254599302,18cyclictest35966irq/58-eth009:10:123
3253299303,21cyclictest0-21swapper/207:30:132
3253299301,23cyclictest26720-21cat11:20:022
3253299301,23cyclictest26720-21cat11:20:012
3251099302,22cyclictest30294-21ssh09:34:120
3254599292,20cyclictest35966irq/58-eth009:15:133
3253299292,21cyclictest2427-21munin-run09:55:022
32532992921,4cyclictest11119-21ssh12:05:112
3253299292,11cyclictest5749-21ssh12:20:112
3253299290,22cyclictest0-21swapper/211:20:132
3253299290,22cyclictest0-21swapper/211:20:132
3251999294,14cyclictest8069-21ssh09:40:141
3251999292,21cyclictest1248-21ssh11:59:311
3251099292,21cyclictest21057-21ssh09:28:570
3254599282,13cyclictest35966irq/58-eth011:50:123
3253299282,11cyclictest0-21swapper/207:53:012
3251099285,16cyclictest12-21ksoftirqd/008:35:120
3251099282,20cyclictest14168-21ssh09:42:010
3254599273,16cyclictest0-21swapper/310:18:013
3254599272,14cyclictest0-21swapper/310:10:143
32545992713,9cyclictest0-21swapper/311:00:143
32545992711,11cyclictest35966irq/58-eth011:35:133
3253299272,20cyclictest0-21swapper/208:50:142
3253299272,10cyclictest18198-21diskmemload09:59:032
3253299272,10cyclictest0-21swapper/210:14:152
3253299271,21cyclictest28918-21ssh10:09:142
3253299271,18cyclictest423-21ls07:10:172
3253299271,15cyclictest8815-21find09:00:112
3253299271,15cyclictest8815-21find09:00:112
3251099275,17cyclictest12-21ksoftirqd/007:25:120
3251099273,18cyclictest12-21ksoftirqd/008:15:140
3251099272,19cyclictest13602-21ssh10:18:010
32510992715,7cyclictest0-21swapper/010:59:130
3251099271,19cyclictest5797-21ssh12:20:140
3254599266,14cyclictest0-21swapper/310:40:143
3251099262,19cyclictest25505-21ssh11:55:140
32510992614,6cyclictest0-21swapper/010:40:140
3251099261,21cyclictest2842-21ssh09:55:140
3254599254,14cyclictest0-21swapper/310:25:143
3254599253,6cyclictest36066irq/59-eth010:05:113
3254599253,15cyclictest0-21swapper/309:55:013
3254599252,17cyclictest0-21swapper/312:29:113
3254599252,17cyclictest0-21swapper/310:39:023
32532992517,5cyclictest12110-21ssh10:35:192
3253299251,3cyclictest18198-21diskmemload12:10:192
3253299251,19cyclictest4752-21ls07:45:222
3251099254,16cyclictest18198-21diskmemload11:15:130
3251099254,16cyclictest18198-21diskmemload11:15:130
3254599243,16cyclictest35966irq/58-eth008:10:103
3254599242,17cyclictest35966irq/58-eth007:55:133
3254599242,13cyclictest35966irq/58-eth007:45:123
32545992412,9cyclictest35966irq/58-eth011:40:143
3254599241,20cyclictest35966irq/58-eth012:30:143
32532992411,10cyclictest17254-21kthreadcore11:50:202
32519992411,10cyclictest2492-21cron08:20:011
3251099241,18cyclictest18625-21rm09:10:150
3254599235,14cyclictest35966irq/58-eth011:45:123
3254599235,11cyclictest0-21swapper/311:30:143
3253299233,6cyclictest0-21swapper/211:41:012
3253299232,16cyclictest0-21swapper/209:44:002
3253299232,16cyclictest0-21swapper/209:39:122
3253299232,16cyclictest0-21swapper/209:29:002
3253299232,16cyclictest0-21swapper/207:45:012
3253299232,15cyclictest0-21swapper/207:35:132
32532992312,9cyclictest771-21ssh09:16:202
32532992312,9cyclictest27268-21ssh11:36:162
32532992311,9cyclictest19734-21ssh12:25:372
32532992311,9cyclictest17341-21ssh11:14:402
3253299230,14cyclictest27346irq/40-dwc2_hso08:35:142
32519992317,3cyclictest0-21swapper/109:15:151
3251099233,3cyclictest131rcu_preempt09:20:130
3251099232,16cyclictest2648-21ssh11:24:160
3251099232,16cyclictest2648-21ssh11:24:160
3251099232,15cyclictest27684-21sh09:15:140
3251099232,15cyclictest20544-21munin-run10:05:000
3251099232,15cyclictest17096-21sh09:45:130
3254599229,7cyclictest35966irq/58-eth011:15:133
3254599229,7cyclictest35966irq/58-eth011:15:123
3254599223,14cyclictest0-21swapper/310:45:163
3254599223,13cyclictest0-21swapper/311:25:123
3254599223,13cyclictest0-21swapper/311:08:013
3254599221,16cyclictest0-21swapper/312:00:013
3253299225,13cyclictest0-21swapper/210:20:172
3253299222,4cyclictest8717-21timerandwakeup07:15:282
3253299222,15cyclictest0-21swapper/212:35:112
3253299222,15cyclictest0-21swapper/210:03:022
3253299222,15cyclictest0-21swapper/209:31:012
3253299221,7cyclictest0-21swapper/208:25:132
32532992213,5cyclictest0-21swapper/212:15:292
32532992211,4cyclictest0-21swapper/208:10:132
3253299221,13cyclictest25387-21ls08:10:012
3251999224,12cyclictest32330-1kworker/1:1H07:40:121
3251999221,16cyclictest26743-21find08:45:111
3254599214,13cyclictest35966irq/58-eth008:00:133
3254599212,9cyclictest0-21swapper/312:10:133
3254599212,12cyclictest35966irq/58-eth010:00:123
32545992116,3cyclictest0-21swapper/307:15:153
32545992111,7cyclictest35966irq/58-eth010:30:133
32545992110,8cyclictest35966irq/58-eth009:25:233
3253299219,9cyclictest12818-21ssh09:24:562
3253299211,6cyclictest91-21pr/ttyS010:35:022
32532992111,8cyclictest0-21swapper/209:10:252
3253299211,16cyclictest28766-21ssh10:45:152
3253299211,15cyclictest0-21swapper/210:53:012
32519992113,5cyclictest26196-21sshd10:06:011
3251999211,17cyclictest2494-21ls09:55:011
3251099212,16cyclictest19463-21apt-get07:30:120
3251099211,9cyclictest27518-21ntp_192.168.11412:30:260
3251099211,14cyclictest18198-21diskmemload10:20:130
3251099211,13cyclictest20811-21cut08:05:170
32510992110,8cyclictest11576-21kthreadcore07:20:200
3254599208,10cyclictest0-21swapper/312:20:113
3254599202,12cyclictest33-21ksoftirqd/311:10:153
3254599201,9cyclictest35966irq/58-eth012:15:103
3253299207,4cyclictest30540-21hwlatdetect08:15:172
3253299202,11cyclictest0-21swapper/209:05:152
3253299201,9cyclictest26866-21fschecks_count08:45:152
3253299201,14cyclictest20652-21wc08:05:022
3253299201,14cyclictest14647-21find07:25:122
3253299201,10cyclictest8237-21usage_EthcTask11:25:322
3253299201,10cyclictest6257-21uname08:20:262
3253299201,10cyclictest6257-21uname08:20:262
3253299201,10cyclictest15814-21ssh11:30:232
3253299201,10cyclictest10630-21kthreadcore07:20:202
32532992010,7cyclictest28412-21ntp_192.168.11410:25:272
32532992010,7cyclictest17949-21nfs4_client10:55:242
3253299200,4cyclictest25339-21kthreadcore11:00:212
3253299200,13cyclictest4232-21ls08:55:182
3251999209,9cyclictest15342-21sed10:20:011
3251999202,15cyclictest18128-21kthreadcore08:35:201
32519992011,6cyclictest7266-21sshd08:21:021
32519992011,6cyclictest7266-21sshd08:21:011
3251999201,16cyclictest14019-21ls10:55:011
3251999201,15cyclictest1585-21kthreadcore08:50:211
32510992012,5cyclictest0-21swapper/008:30:160
32510992010,8cyclictest13383-21cron09:05:020
32510992010,8cyclictest13383-21cron09:05:010
3254599193,11cyclictest0-21swapper/310:53:013
3254599192,10cyclictest0-21swapper/309:00:133
3254599192,10cyclictest0-21swapper/309:00:133
3254599192,10cyclictest0-21swapper/307:20:133
32545991911,3cyclictest0-21swapper/309:35:143
3254599191,12cyclictest0-21swapper/307:30:163
3254599191,11cyclictest0-21swapper/308:05:163
3253299191,9cyclictest22512-21sed08:40:192
3253299191,9cyclictest14275-21kthreadcore08:30:202
3253299191,8cyclictest11160-21ls07:55:132
3251999199,7cyclictest27287-21cut11:00:291
3251999197,9cyclictest7843-21sed11:45:191
3251999197,9cyclictest31254-21cat11:20:291
3251999197,9cyclictest31254-21cat11:20:291
3251999197,3cyclictest18198-21diskmemload10:01:581
3251999192,14cyclictest21464-21latency_hist11:35:011
3251999192,11cyclictest0-21swapper/107:35:131
32519991912,4cyclictest4257-21/usr/sbin/munin09:20:141
32519991911,5cyclictest0-21swapper/108:40:141
3251999191,14cyclictest8830-21apt-get09:00:121
3251999191,14cyclictest8830-21apt-get09:00:121
3251999191,10cyclictest385-21tail11:05:171
3251999191,10cyclictest23182-21missed_timers10:40:221
3251099193,9cyclictest0-21swapper/011:08:020
3251099192,13cyclictest844-21munin-node12:05:120
3251099191,16cyclictest15925-21apt-get08:00:110
3254599183,10cyclictest0-21swapper/307:45:013
3254599182,10cyclictest0-21swapper/308:35:143
3254599182,10cyclictest0-21swapper/308:15:143
3254599182,10cyclictest0-21swapper/307:25:153
3254599181,13cyclictest0-21swapper/308:20:153
3254599181,13cyclictest0-21swapper/308:20:153
3254599181,12cyclictest0-21swapper/312:35:303
3253299181,8cyclictest32566-21latency_hist07:10:022
3251999189,7cyclictest26019-21ps09:30:251
3251999189,7cyclictest18186-21cron09:10:011
3251999188,7cyclictest4059-21cut09:55:211
3251999188,7cyclictest16312-21cut10:20:171
3251999185,9cyclictest22460-21ssh11:15:301
3251999185,9cyclictest22460-21ssh11:15:301
3251999185,4cyclictest131rcu_preempt12:05:181
3251999185,3cyclictest131rcu_preempt11:50:181
3251999182,11cyclictest0-21swapper/108:00:131
3251999181,8cyclictest2346-21cut10:30:191
3251999181,8cyclictest21178-21unixbench_singl09:45:281
32519991812,4cyclictest0-21swapper/107:45:301
3251999181,14cyclictest16666-21kthreadcore10:55:211
3251999181,13cyclictest5717-21ntp_192.168.11412:00:241
3251099189,7cyclictest13618-21apt-get09:05:140
3251099189,6cyclictest4101-21grep07:10:270
3251099183,7cyclictest0-21swapper/008:10:110
3251099183,10cyclictest0-21swapper/007:15:140
3251099182,11cyclictest0-21swapper/008:20:150
3251099182,11cyclictest0-21swapper/008:20:140
3251099181,4cyclictest131rcu_preempt11:45:130
3251099181,14cyclictest1416-21cut09:35:210
32510991810,6cyclictest31299-21sshd08:48:010
3254599172,10cyclictest0-21swapper/308:50:043
3254599172,10cyclictest0-21swapper/308:45:143
3254599172,10cyclictest0-21swapper/307:50:043
3254599171,13cyclictest0-21swapper/307:10:153
3254599171,12cyclictest0-21swapper/308:40:223
3254599171,11cyclictest0-21swapper/307:35:143
32545991710,4cyclictest0-21swapper/309:40:263
3251999177,8cyclictest3680-21apt-get11:25:131
3251999175,9cyclictest29346-21irqrtprio10:45:191
3251999174,8cyclictest21779-21find11:35:121
3251999173,8cyclictest0-21swapper/112:20:121
3251999172,12cyclictest14933-21ssh12:25:151
3251999172,10cyclictest0-21swapper/112:19:381
3251999172,10cyclictest0-21swapper/107:55:131
3251999171,8cyclictest0-21swapper/112:39:411
3251999171,13cyclictest23787-21ntp_192.168.11412:10:241
3251999171,12cyclictest18596-21apt-get09:10:121
3251999171,12cyclictest14406-21ssh10:35:261
32519991710,4cyclictest29966-21/usr/sbin/munin10:10:281
3251099171,13cyclictest407-21kthreadcore08:50:200
3251099171,13cyclictest32447-21ntp_192.168.11407:40:230
3251099170,5cyclictest15887-21latency_hist08:00:020
3254599162,9cyclictest0-21swapper/308:25:133
3254599162,6cyclictest0-21swapper/309:10:003
3251999167,6cyclictest7757-21timerwakeupswit08:55:301
3251999166,7cyclictest8812-21cut08:25:201
3251999166,7cyclictest21-21ksoftirqd/107:25:181
3251999162,12cyclictest6878-21sed07:50:191
3251999161,9cyclictest5512-21ps07:15:191
3251999161,12cyclictest3348-21cut07:10:211
3251999161,12cyclictest22848-21ntp_192.168.11407:30:231
3251999161,12cyclictest22830-21kthreadcore08:05:211
3251999161,11cyclictest9303-21perl11:10:181
32519991610,4cyclictest29094-21sendmail_mailtr08:10:261
3251099168,6cyclictest3162-21cut07:45:210
32510991610,4cyclictest24918-21grep07:35:190
3251999151,11cyclictest9871-21apt-get07:20:131
3251999141,9cyclictest32604-21latency_hist07:10:031
3251099141,10cyclictest32518-21date07:10:010
3254599132,8cyclictest0-21swapper/308:55:283
3254599131,7cyclictest0-21swapper/308:30:163
3254599111,7cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional