You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-06 - 21:17

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot4s.osadl.org (updated Sun Sep 06, 2026 12:45:11)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
28968997841,26cyclictest35966irq/58-eth011:10:123
2896099601,35cyclictest19070-21ssh09:30:142
28968994730,10cyclictest35966irq/58-eth010:25:123
2896099461,31cyclictest15001-21diskmemload09:25:102
2894499431,22cyclictest22760-21ssh09:50:120
28952994219,18cyclictest21-21ksoftirqd/111:30:121
2896899412,24cyclictest35966irq/58-eth010:20:133
2896899401,24cyclictest35966irq/58-eth007:10:123
2896099402,23cyclictest1283-21ssh12:20:142
2896099401,28cyclictest15439-21ssh09:10:162
2896899382,23cyclictest35966irq/58-eth012:15:133
2896099383,27cyclictest15705-21ssh10:40:132
28968993714,13cyclictest35966irq/58-eth010:40:143
2894499373,25cyclictest8350-21ssh10:00:130
2896099353,24cyclictest0-21swapper/212:25:242
2896099353,24cyclictest0-21swapper/212:25:232
2896099344,19cyclictest0-21swapper/211:35:132
2894499344,26cyclictest26539-21ssh11:40:130
28968993217,9cyclictest35966irq/58-eth010:55:143
2896099324,19cyclictest27346irq/40-dwc2_hso07:35:132
2896099323,10cyclictest0-21swapper/212:30:172
28960993222,5cyclictest15737-21sh12:10:132
2896099322,19cyclictest12023-21ssh10:20:132
2896099315,17cyclictest27346irq/40-dwc2_hso08:00:132
2896099312,24cyclictest21113-21ssh11:55:152
2896099303,18cyclictest27346irq/40-dwc2_hso08:45:122
2896099302,21cyclictest0-21swapper/207:10:132
28960993015,5cyclictest27-21ksoftirqd/212:09:282
28960993011,15cyclictest1-21systemd12:15:162
2895299301,23cyclictest15734-21ssh12:10:121
2894499303,21cyclictest19441-21ssh11:00:150
2896899292,16cyclictest321rcuc/312:25:123
2896899292,16cyclictest321rcuc/312:25:123
2896899292,15cyclictest35966irq/58-eth011:20:133
2896099296,11cyclictest27-21ksoftirqd/211:07:512
2896099293,17cyclictest27346irq/40-dwc2_hso08:50:132
2896099292,18cyclictest27346irq/40-dwc2_hso09:00:132
28960992915,10cyclictest927-21/usr/sbin/munin09:20:282
28960992912,8cyclictest27-21ksoftirqd/210:00:252
2895299291,17cyclictest31751-21ssh09:55:121
2896899281,17cyclictest35966irq/58-eth007:30:103
2896099283,15cyclictest27346irq/40-dwc2_hso08:10:122
2896099282,9cyclictest0-21swapper/209:05:132
2896099282,16cyclictest27346irq/40-dwc2_hso10:30:132
2896099282,11cyclictest0-21swapper/207:40:052
28960992811,14cyclictest261rcuc/211:25:112
2894499281,19cyclictest28400-21ssh11:05:140
2896899276,13cyclictest321rcuc/309:40:123
2896099278,4cyclictest27-21ksoftirqd/209:55:522
2896099272,10cyclictest0-21swapper/209:40:142
2896099271,20cyclictest28099-21ssh09:35:122
2896099271,17cyclictest26745-21uniq11:40:182
2896099271,14cyclictest31553-21sh09:55:032
2895299271,19cyclictest989-21ssh09:20:141
2896099264,15cyclictest27346irq/40-dwc2_hso07:30:122
2896099263,12cyclictest6650-21ssh10:35:152
2896099260,8cyclictest26441-21ssh10:10:152
2895299261,7cyclictest17749-21perl11:35:161
2895299261,14cyclictest21-21ksoftirqd/109:15:141
2894499263,7cyclictest131rcu_preempt11:30:130
2894499263,19cyclictest1268-21ssh12:20:130
2894499261,19cyclictest6752-21ssh12:05:140
28968992512,6cyclictest35966irq/58-eth012:35:163
2896899251,14cyclictest35966irq/58-eth009:25:113
2896099252,16cyclictest0-21swapper/212:35:172
28960992515,8cyclictest0-21swapper/211:10:142
2896099250,5cyclictest11206-21acpi07:25:112
2895299256,13cyclictest0-21swapper/111:40:161
2895299253,15cyclictest3094-21ssh11:45:111
28968992411,7cyclictest35966irq/58-eth010:45:133
2896899241,12cyclictest35966irq/58-eth007:20:133
2896099245,16cyclictest9901-21kthreadcore07:55:202
2896099244,12cyclictest27346irq/40-dwc2_hso10:15:132
2896099243,14cyclictest27346irq/40-dwc2_hso08:55:142
2896099243,14cyclictest27346irq/40-dwc2_hso08:55:142
2896099241,12cyclictest16095-21sort11:50:282
28960992411,10cyclictest19217-21ssh09:46:152
2895299244,15cyclictest28282-21ssh12:35:161
2894499242,19cyclictest24397-21ssh09:15:140
2896899235,11cyclictest35966irq/58-eth011:15:143
2896899232,13cyclictest35966irq/58-eth009:10:143
2896099236,13cyclictest0-21swapper/207:20:012
2896099235,4cyclictest27-21ksoftirqd/210:25:322
2896099232,15cyclictest27346irq/40-dwc2_hso10:55:132
2896099232,12cyclictest30122-21ssh12:00:132
2896099232,11cyclictest23391-21xargs11:00:312
2896099232,10cyclictest18040-21irqrtprio08:05:172
2896099231,18cyclictest32067-21tune2fs08:20:152
2894499231,19cyclictest10248-21apt-get12:25:130
2894499231,19cyclictest10248-21apt-get12:25:120
2896899222,13cyclictest35966irq/58-eth008:10:113
28968992211,8cyclictest0-21swapper/310:00:283
2896899221,14cyclictest0-21swapper/309:45:133
2896099229,5cyclictest261rcuc/208:25:122
2896099228,7cyclictest13861-21apt-get08:35:162
2896099225,4cyclictest27-21ksoftirqd/210:05:272
2896099225,14cyclictest25467-21kernelversion10:45:182
2896099224,15cyclictest30743-21fschecks_time07:45:162
2896099222,13cyclictest27346irq/40-dwc2_hso10:50:132
2896099222,11cyclictest27446-21kthreadcore09:15:222
2896099222,10cyclictest15184-21kthreadcore11:15:192
2896099221,12cyclictest9326-21sed11:30:192
2896099221,12cyclictest3753-21irqrtprio11:45:172
2896099221,10cyclictest31734-21latency_hist08:20:022
2896099221,10cyclictest23142-21sort11:20:162
2895299221,17cyclictest19440-21sh11:00:151
2895299221,15cyclictest2995-21cpuspeed07:50:131
2894499223,16cyclictest17378-21ssh10:05:120
2894499221,14cyclictest16023-21find07:30:100
2894499220,16cyclictest20831-21find07:35:110
2894499220,15cyclictest13721-21ssh09:45:140
2896899219,9cyclictest0-21swapper/311:05:423
2896899218,8cyclictest321rcuc/308:35:153
2896899212,13cyclictest35966irq/58-eth012:00:123
28968992112,6cyclictest0-21swapper/311:58:183
2896099213,11cyclictest27346irq/40-dwc2_hso08:40:152
2896099213,11cyclictest27346irq/40-dwc2_hso08:40:142
28960992112,7cyclictest0-21swapper/207:50:222
2896099211,10cyclictest8011-21kthreadcore07:20:212
2896099210,11cyclictest10832-21kthreadcore08:30:192
2895299211,7cyclictest15523-21/usr/sbin/munin11:50:231
2895299211,6cyclictest18593-21apt-get09:30:021
2895299210,18cyclictest24984-21ls12:15:171
2894499214,3cyclictest131rcu_preempt10:50:260
2894499211,17cyclictest6153-21apt-config07:20:010
28944992111,7cyclictest11637-21basename10:20:000
2894499211,15cyclictest6179-21ls07:50:210
2896899203,12cyclictest0-21swapper/310:50:133
28968992014,4cyclictest0-21swapper/309:30:153
28968992011,7cyclictest0-21swapper/312:10:103
28968992011,6cyclictest35966irq/58-eth012:20:143
2896899201,13cyclictest35966irq/58-eth010:15:133
2896899201,13cyclictest0-21swapper/307:25:123
2896899201,12cyclictest0-21swapper/311:25:133
2896099201,16cyclictest29073-21latency_hist07:10:032
2895299204,3cyclictest131rcu_preempt12:05:191
28952992012,6cyclictest1-21systemd10:50:081
28952992012,5cyclictest10557-21cut10:55:161
2895299201,17cyclictest28785-21timerwakeupswit10:45:291
2895299201,16cyclictest20994-21apt-get10:25:111
2895299201,14cyclictest31618-21latency_hist08:20:011
2895299201,14cyclictest17601-21timerwakeupswit08:35:311
2894499205,6cyclictest12-21ksoftirqd/008:20:110
2894499201,9cyclictest2728066ssh09:34:320
2894499201,6cyclictest15779-21ntp_states11:50:260
2894499201,14cyclictest988-21sh09:20:150
2894499201,14cyclictest31966-21ssh11:25:110
2896899199,7cyclictest0-21swapper/311:33:423
2896899193,9cyclictest0-21swapper/312:30:183
2896899193,10cyclictest35966irq/58-eth007:50:113
2896899192,10cyclictest0-21swapper/308:15:133
2896899192,10cyclictest0-21swapper/307:35:143
28968991913,4cyclictest0-21swapper/311:45:273
28968991913,4cyclictest0-21swapper/310:05:193
28968991913,4cyclictest0-21swapper/309:20:183
2896899191,14cyclictest0-21swapper/309:55:023
2895299199,7cyclictest5407-21kthreadcore08:25:191
2895299197,9cyclictest4769-21df09:40:141
2895299191,9cyclictest13984-21/usr/sbin/munin11:15:121
28952991912,4cyclictest21183-21sendmail_mailst10:05:271
28952991911,5cyclictest23138-21cpuspeed11:20:151
28952991910,6cyclictest1-21systemd12:20:161
2894499197,8cyclictest0-21swapper/008:05:140
2894499195,3cyclictest131rcu_preempt10:35:160
2894499193,4cyclictest12-21ksoftirqd/012:03:540
2894499192,4cyclictest9919-21apt-get09:25:110
28944991912,5cyclictest0-21swapper/009:40:260
28944991912,4cyclictest0-21swapper/007:55:120
2894499191,16cyclictest18175-21kthreadcore10:40:230
2894499191,14cyclictest3115-21sh11:45:140
2894499191,13cyclictest8588-21sh11:10:240
2894499191,13cyclictest24743-21ssh11:55:260
2894499191,10cyclictest0-21swapper/011:36:000
2896899182,10cyclictest0-21swapper/308:55:053
2896899182,10cyclictest0-21swapper/308:55:043
2896899182,10cyclictest0-21swapper/308:50:133
2896899182,10cyclictest0-21swapper/308:00:133
2896899182,10cyclictest0-21swapper/307:55:143
2896899182,10cyclictest0-21swapper/307:40:043
28968991813,3cyclictest0-21swapper/312:05:193
28968991812,4cyclictest0-21swapper/311:50:273
28968991812,4cyclictest0-21swapper/311:40:263
28968991812,4cyclictest0-21swapper/311:00:113
28968991812,4cyclictest0-21swapper/310:35:163
28968991812,4cyclictest0-21swapper/310:12:543
28968991812,4cyclictest0-21swapper/309:55:283
28968991812,4cyclictest0-21swapper/309:15:273
2896899181,12cyclictest0-21swapper/307:15:173
28968991810,6cyclictest0-21swapper/311:35:153
2895299188,7cyclictest28312-21apt-get08:50:111
2895299182,11cyclictest0-21swapper/107:20:051
2895299181,8cyclictest32087-21ntp_states11:05:261
28952991812,4cyclictest0-21swapper/110:02:091
2895299181,14cyclictest21093-21ssh11:55:121
28952991810,5cyclictest26377-21ntp_kernel_err09:50:251
2895299180,3cyclictest131rcu_preempt09:30:021
2895299180,16cyclictest13830-21ls11:15:011
2895299180,12cyclictest12522-21ls08:00:021
2894499188,2cyclictest12-21ksoftirqd/010:55:560
2894499186,9cyclictest24151-21kthreadcore08:10:190
2894499184,3cyclictest131rcu_preempt10:20:180
2894499181,4cyclictest0-21swapper/010:28:430
2894499181,3cyclictest131rcu_preempt12:17:430
2894499181,3cyclictest131rcu_preempt10:33:170
2894499181,14cyclictest17367-21kthreadcore09:10:230
2894499181,11cyclictest31726-21ssh09:55:120
28944991810,6cyclictest17363-1kworker/u9:112:30:170
2894499180,12cyclictest1096-21ls10:50:010
2896899179,6cyclictest0-21swapper/309:35:243
2896899178,7cyclictest321rcuc/308:30:123
2896899172,10cyclictest0-21swapper/308:40:153
2896899172,10cyclictest0-21swapper/308:40:143
28968991711,4cyclictest0-21swapper/309:00:173
2896899171,10cyclictest0-21swapper/310:30:143
2895299179,6cyclictest19999-21cat10:40:301
2895299178,6cyclictest5411-21unixbench_singl07:15:301
2895299178,6cyclictest1198-21kthreadcore11:25:201
2895299176,3cyclictest131rcu_preempt09:45:211
2895299172,7cyclictest131rcu_preempt10:24:471
2895299172,13cyclictest31355-21kthreadcore09:35:231
2895299172,12cyclictest30036-21ssh10:30:141
2895299171,9cyclictest11323-21forks07:25:161
2895299171,7cyclictest19545-21sort12:30:171
2895299171,7cyclictest16031-21ntp_kernel_err08:00:241
2895299171,3cyclictest131rcu_preempt12:25:191
2895299171,3cyclictest131rcu_preempt12:25:181
2895299171,14cyclictest1-21systemd07:13:121
2895299171,13cyclictest26559-21ssh10:10:171
2895299171,13cyclictest20110-21usage_EthcTask09:10:331
2895299171,10cyclictest21021-21grep07:35:161
2895299171,10cyclictest20735-21missed_timers08:05:221
28952991710,5cyclictest3412-21irqcore10:15:171
2895299170,8cyclictest2617-21cut08:20:221
2894499179,6cyclictest10261-21grep07:20:280
2894499172,3cyclictest131rcu_preempt12:14:260
2894499172,10cyclictest0-21swapper/008:35:160
2894499171,3cyclictest0-21swapper/009:37:250
2894499171,13cyclictest28385-21df_inode12:35:170
2894499171,11cyclictest14024-21ssh11:15:120
2896899162,9cyclictest0-21swapper/308:45:143
2896899162,9cyclictest0-21swapper/307:45:143
2895299169,5cyclictest27016-21grep08:45:261
2895299168,6cyclictest1785-21ls07:45:271
2895299164,9cyclictest32008-21cut12:00:221
2895299164,10cyclictest8425-21kthreadcore10:35:211
28952991611,3cyclictest968-21fschecks_time08:55:181
28952991611,3cyclictest968-21fschecks_time08:55:171
2895299161,12cyclictest6001-21cut09:00:171
2895299161,11cyclictest9106-21cpuspeed08:30:141
2895299161,11cyclictest29631-21grep07:40:291
2895299161,11cyclictest22206-21ps08:40:281
2895299161,11cyclictest22206-21ps08:40:271
28952991610,4cyclictest16212-21fschecks_time07:30:141
2894499169,5cyclictest32630-21awk07:10:250
2894499168,6cyclictest8803-21sshd08:29:080
2894499168,6cyclictest29007-21sed07:10:010
2894499165,8cyclictest9469-21irqcore08:30:160
2894499164,10cyclictest30530-21grep10:10:300
2894499163,11cyclictest26909-21kthreadcore07:40:200
2894499161,7cyclictest30585-21sendmail_mailqu08:15:270
2894499161,13cyclictest15755-21kthreadcore08:00:210
2894499161,12cyclictest761-21dpkg08:55:050
2894499161,12cyclictest761-21dpkg08:55:040
2894499161,12cyclictest26914-21ntp_192.168.11408:45:240
2894499161,11cyclictest31795-21ntp_states08:50:250
28944991610,4cyclictest2706-21sh07:50:010
28944991610,3cyclictest0-21swapper/011:20:130
2895299151,11cyclictest10449-21apt-get09:05:141
2895299151,10cyclictest23235-21kthreadcore08:10:191
2894499151,11cyclictest7694-21tr09:00:210
2894499151,11cyclictest20688-21kthreadcore08:40:210
2894499151,11cyclictest20688-21kthreadcore08:40:210
2894499151,11cyclictest10767-21tail09:05:180
2896899148,4cyclictest0-21swapper/309:05:123
2894499141,10cyclictest11205-21/usr/sbin/munin07:25:110
2896899137,4cyclictest0-21swapper/308:05:243
2896899131,9cyclictest0-21swapper/308:30:013
2895299130,10cyclictest28959-21sed07:10:011
2896899123,7cyclictest0-21swapper/308:20:273
2896899101,6cyclictest0-21swapper/307:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional