You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-20 - 21:03

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot4s.osadl.org (updated Wed May 20, 2026 12:45:14)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3601996038,14cyclictest27-21ksoftirqd/210:10:112
359399592,38cyclictest24117-21ssh10:05:121
359399592,38cyclictest24117-21ssh10:05:121
3601995815,7cyclictest131rcu_preempt07:40:122
3613995641,10cyclictest35966irq/58-eth011:10:133
3593995419,23cyclictest21-21ksoftirqd/107:50:131
3613995231,14cyclictest0-21swapper/309:20:113
3601995225,18cyclictest131rcu_preempt08:15:132
360199480,30cyclictest30780-21ssh12:15:112
361399477,21cyclictest35966irq/58-eth010:25:123
359399461,6cyclictest131rcu_preempt12:10:131
361399444,27cyclictest35966irq/58-eth009:55:123
361399444,27cyclictest35966irq/58-eth009:55:123
359399441,32cyclictest11439-21ssh11:10:131
361399435,23cyclictest35966irq/58-eth010:10:133
359399431,28cyclictest12868-21ssh12:05:121
361399391,23cyclictest35966irq/58-eth011:50:113
3601993921,7cyclictest27-21ksoftirqd/211:20:142
359399378,8cyclictest131rcu_preempt07:45:121
359399376,17cyclictest15119-21ssh10:00:131
359399374,22cyclictest2452-21ssh11:05:131
3593993721,10cyclictest3580-21cyclictest10:35:111
358899373,10cyclictest131rcu_preempt08:40:130
361399365,22cyclictest35966irq/58-eth011:55:133
358899364,29cyclictest20354-21ssh11:15:120
358899361,29cyclictest1839-21ssh12:35:120
361399353,23cyclictest35966irq/58-eth007:15:133
360199352,25cyclictest0-21swapper/207:25:142
3601993518,12cyclictest0-21swapper/212:05:132
360199351,26cyclictest0-21swapper/210:40:112
358899351,30cyclictest18291-21ssh11:50:120
360199341,24cyclictest7309-21ssh12:20:122
359399347,14cyclictest3768-21ssh12:00:131
359399342,18cyclictest27217-21diskmemload11:55:131
358899342,27cyclictest16782-21rm09:25:110
358899342,26cyclictest9377-21ssh11:45:120
360199332,22cyclictest27-21ksoftirqd/209:15:112
3601993313,5cyclictest17264-21fschecks_count09:05:162
359399332,21cyclictest27728-21ssh10:25:121
361399323,21cyclictest35966irq/58-eth011:05:133
3588993224,6cyclictest12-21ksoftirqd/012:20:120
358899321,24cyclictest8017-21ssh10:50:140
3613993112,9cyclictest35966irq/58-eth009:50:133
3613993112,9cyclictest35966irq/58-eth009:50:133
358899319,12cyclictest131rcu_preempt07:40:130
358899315,4cyclictest131rcu_preempt11:55:140
358899312,25cyclictest7847-21ssh09:20:120
3613993017,9cyclictest35966irq/58-eth010:40:113
361399301,20cyclictest35966irq/58-eth012:00:133
359399302,23cyclictest13261-21ssh09:58:191
359399302,23cyclictest13261-21ssh09:58:181
358899304,21cyclictest23897-21ssh11:35:140
3601992916,8cyclictest27-21ksoftirqd/209:58:192
3601992916,8cyclictest27-21ksoftirqd/209:58:182
3601992912,11cyclictest19715-21kthreadcore10:20:202
3588992916,11cyclictest12-21ksoftirqd/010:25:120
3588992913,9cyclictest131rcu_preempt09:55:120
3588992913,9cyclictest131rcu_preempt09:55:120
3588992910,2cyclictest131rcu_preempt08:25:140
361399287,10cyclictest0-21swapper/310:35:113
361399283,16cyclictest0-21swapper/309:41:113
360199281,6cyclictest19690-21unixbench_singl08:30:282
359399282,19cyclictest25343-21ssh12:30:141
359399282,19cyclictest25343-21ssh12:30:141
361399273,17cyclictest0-21swapper/309:18:113
361399273,16cyclictest0-21swapper/311:26:113
360199271,23cyclictest14903-21ssh11:30:122
360199271,19cyclictest0-21swapper/207:30:142
360199271,17cyclictest18279-21ssh11:50:112
360199270,17cyclictest27346irq/40-dwc2_hso09:35:142
359399272,19cyclictest16955-21sh09:41:111
359399272,19cyclictest11363-21sh11:26:111
359399272,16cyclictest30798-21sh12:15:121
359399271,18cyclictest4495-21ssh10:30:131
358899273,19cyclictest345-21sh11:40:130
358899273,16cyclictest12-21ksoftirqd/009:05:130
358899272,21cyclictest31191-21ssh09:15:120
360199264,18cyclictest5940-21ssh11:25:122
360199262,9cyclictest27764-21apt-get07:35:112
360199262,9cyclictest25814-21apt-get09:30:122
360199262,15cyclictest24216-21apt-get08:05:122
3601992620,4cyclictest27-21ksoftirqd/210:50:122
359399261,8cyclictest0-21swapper/108:25:141
3593992610,9cyclictest3580-21cyclictest08:00:121
358899264,3cyclictest131rcu_preempt07:45:120
358899263,17cyclictest14918-21ssh11:30:130
358899263,14cyclictest16938-21ssh10:55:130
358899263,14cyclictest12-21ksoftirqd/009:00:140
358899261,21cyclictest9668-21ssh10:15:130
361399253,14cyclictest0-21swapper/311:15:133
3613992510,9cyclictest35966irq/58-eth012:35:133
360199251,14cyclictest6017-21cat11:05:242
360199250,17cyclictest24331-21grep10:05:152
360199250,17cyclictest24331-21grep10:05:152
359399251,10cyclictest0-21swapper/110:55:211
358899253,4cyclictest131rcu_preempt07:20:130
3588992512,10cyclictest5675-21cron09:55:010
3588992512,10cyclictest5675-21cron09:55:010
361399242,10cyclictest0-21swapper/310:20:143
360199247,11cyclictest27-21ksoftirqd/211:15:252
360199243,8cyclictest21715-21kworker/2:109:14:482
360199242,5cyclictest23031-21sort08:00:272
360199242,18cyclictest20535-21ssh09:45:132
360199241,5cyclictest0-21swapper/208:10:022
3601992410,5cyclictest9955-21fschecks_time07:50:162
359399241,17cyclictest7485-21apt-get08:55:131
359399241,17cyclictest22471-21sh10:40:121
358899242,19cyclictest19263-21sh11:14:110
358899242,18cyclictest29543-21ssh09:30:250
361399232,14cyclictest35966irq/58-eth010:45:133
3613992310,10cyclictest0-21swapper/310:50:223
360199237,3cyclictest131rcu_preempt10:00:242
360199231,8cyclictest7274-21uname07:10:282
360199231,6cyclictest5484-21kthreadcore10:30:192
3601992316,6cyclictest27346irq/40-dwc2_hso09:20:112
360199231,5cyclictest32309-21cat10:25:302
3601992311,9cyclictest23156-21ssh10:57:142
360199231,14cyclictest9380-21sh11:45:122
3601992310,6cyclictest19946-21cron11:15:012
3601992310,10cyclictest4489-21kernelversion12:00:192
359399239,11cyclictest1146-21cron08:15:011
359399231,17cyclictest26221-21ls07:30:221
359399231,12cyclictest30983-21kthreadcore09:50:201
359399231,12cyclictest30983-21kthreadcore09:50:201
358899237,13cyclictest0-21swapper/010:20:140
358899231,3cyclictest131rcu_preempt09:43:480
358899231,3cyclictest131rcu_preempt09:14:480
358899231,2cyclictest131rcu_preempt10:07:470
358899231,2cyclictest131rcu_preempt10:07:470
361399223,12cyclictest0-21swapper/312:05:133
360199221,9cyclictest0-21swapper/209:40:212
360199221,9cyclictest0-21swapper/208:40:132
360199221,4cyclictest2854466ssh11:00:242
360199221,3cyclictest0-21swapper/209:50:262
360199221,3cyclictest0-21swapper/209:50:262
3601992210,9cyclictest3975-21ssh10:45:392
3601992210,9cyclictest32314-21ssh11:39:552
3601992210,5cyclictest7117-21ssh11:42:432
359399222,14cyclictest26632-21ssh11:17:111
3593992212,7cyclictest7901-21cpuspeed09:20:121
358899222,17cyclictest28979-21ssh12:30:260
358899222,17cyclictest28979-21ssh12:30:260
3588992215,5cyclictest12-21ksoftirqd/009:45:130
358899221,3cyclictest131rcu_preempt11:25:150
3588992213,7cyclictest13093-21cron07:20:000
3588992212,5cyclictest0-21swapper/011:00:150
358899221,16cyclictest4253-21ssh10:10:230
358899221,14cyclictest20050-21ssh12:25:270
361399218,9cyclictest0-21swapper/312:10:193
3613992113,4cyclictest0-21swapper/311:45:113
361399211,12cyclictest35966irq/58-eth008:05:123
360199219,9cyclictest0-21swapper/207:15:162
360199218,6cyclictest13670-21grep07:20:172
360199215,14cyclictest628-21tcf-agent10:15:122
360199215,12cyclictest27-21ksoftirqd/209:00:142
360199213,15cyclictest27346irq/40-dwc2_hso07:45:132
360199211,8cyclictest30461-21cut08:45:152
360199211,8cyclictest0-21swapper/207:55:132
360199211,3cyclictest0-21swapper/212:35:452
3601992111,8cyclictest0-21swapper/212:25:282
360199211,15cyclictest13434-21ssh10:35:102
360199211,15cyclictest0-21swapper/211:55:142
360199211,11cyclictest14704-21cat08:25:292
3601992110,8cyclictest0-21swapper/212:10:162
360199210,5cyclictest19282-21kthreadcore09:25:192
359399211,3cyclictest131rcu_preempt09:30:121
359399211,17cyclictest25640-21ls09:10:221
359399211,17cyclictest20468-21ls09:05:231
3593992110,7cyclictest11593-21sh10:50:241
358899219,3cyclictest131rcu_preempt07:35:130
358899218,9cyclictest12-21ksoftirqd/010:45:120
358899211,15cyclictest26159-21ssh10:40:260
358899210,14cyclictest9931-21cut07:50:160
361399208,9cyclictest0-21swapper/311:20:293
361399202,11cyclictest0-21swapper/309:25:113
3613992011,7cyclictest0-21swapper/310:57:213
361399201,11cyclictest0-21swapper/312:15:123
360199206,11cyclictest2862-21fschecks_time08:50:152
360199201,9cyclictest12097-21munin-plugin-st09:00:012
360199201,15cyclictest28853-21missed_timers12:30:242
360199201,15cyclictest28853-21missed_timers12:30:242
359399202,15cyclictest21725-21diskmemload09:15:111
359399202,13cyclictest16330-21ssh12:25:141
359399201,16cyclictest28791-21ls08:10:011
359399201,10cyclictest0-21swapper/109:00:231
358899208,9cyclictest26708-21cat12:10:320
358899205,9cyclictest12-21ksoftirqd/008:15:130
358899203,12cyclictest0-21swapper/007:55:150
358899202,15cyclictest30811-21ssh12:15:130
358899201,10cyclictest16505-21grep08:30:180
358899200,5cyclictest131rcu_preempt08:55:140
358899200,2cyclictest131rcu_preempt10:30:020
361399199,8cyclictest0-21swapper/311:31:433
361399199,8cyclictest0-21swapper/309:36:363
361399192,9cyclictest0-21swapper/307:10:013
361399192,12cyclictest0-21swapper/309:34:113
3613991911,5cyclictest35966irq/58-eth011:35:233
361399191,12cyclictest35966irq/58-eth010:05:113
361399191,12cyclictest35966irq/58-eth010:05:113
361399191,12cyclictest0-21swapper/308:00:223
361399191,12cyclictest0-21swapper/307:55:133
361399191,11cyclictest0-21swapper/308:30:013
3613991910,6cyclictest0-21swapper/312:23:363
3613991910,6cyclictest0-21swapper/311:43:213
360199191,9cyclictest21216-21irqrtprio08:35:162
360199191,10cyclictest3609-21latency_hist07:10:012
359399192,12cyclictest23872-21ssh11:35:121
359399191,8cyclictest6046-21grep12:35:311
359399191,7cyclictest30460-21grep08:45:151
359399191,4cyclictest29535-21ntp_192.168.11411:00:271
3593991912,4cyclictest0-21swapper/107:20:261
3593991911,5cyclictest11109-21cron09:40:011
3593991911,4cyclictest31378-21kthreadcore11:20:211
359399191,14cyclictest25525-21apt-get08:40:131
359399190,14cyclictest7386-21ls07:10:291
358899196,3cyclictest131rcu_preempt12:09:080
358899193,14cyclictest19442-21kthreadcore07:25:190
358899193,12cyclictest12-21ksoftirqd/007:10:140
358899192,3cyclictest131rcu_preempt09:35:140
358899191,15cyclictest15195-21/usr/sbin/munin10:00:130
3588991910,6cyclictest18847-21sshd10:36:110
361399189,6cyclictest0-21swapper/310:30:283
361399189,6cyclictest0-21swapper/310:16:493
361399183,11cyclictest0-21swapper/309:49:113
361399183,11cyclictest0-21swapper/309:10:053
361399182,10cyclictest0-21swapper/308:55:143
361399182,10cyclictest0-21swapper/308:30:143
361399182,10cyclictest0-21swapper/307:20:133
361399181,13cyclictest0-21swapper/312:25:143
361399181,12cyclictest35966irq/58-eth008:35:103
361399181,11cyclictest0-21swapper/309:00:143
361399181,11cyclictest0-21swapper/307:25:163
3613991810,6cyclictest0-21swapper/312:31:223
3613991810,6cyclictest0-21swapper/312:31:213
3613991810,6cyclictest0-21swapper/311:00:593
360199181,8cyclictest6339-21fschecks_count08:20:152
360199181,8cyclictest6339-21fschecks_count08:20:152
359399184,7cyclictest131rcu_preempt11:40:121
359399184,6cyclictest131rcu_preempt08:30:201
359399183,6cyclictest22405-21ntp_states10:20:271
359399182,12cyclictest703-21apt-get10:10:121
359399181,6cyclictest7401-21df12:20:131
359399181,4cyclictest18275-21munin-run10:20:011
359399181,2cyclictest131rcu_preempt09:45:121
359399181,14cyclictest310-21fschecks_time07:40:161
359399181,12cyclictest31441-21ssh10:45:131
3593991810,5cyclictest0-21swapper/108:20:261
3593991810,5cyclictest0-21swapper/108:20:261
358899183,13cyclictest4624-21kthreadcore08:50:200
358899181,3cyclictest131rcu_preempt08:35:110
358899181,2cyclictest131rcu_preempt11:05:140
3588991812,4cyclictest0-21swapper/008:10:150
358899181,14cyclictest24271-21ls08:05:140
358899181,13cyclictest3757-21apt-get12:00:140
361399179,6cyclictest0-21swapper/310:03:093
361399172,10cyclictest0-21swapper/308:40:143
361399172,10cyclictest0-21swapper/308:15:143
361399172,10cyclictest0-21swapper/307:30:143
359399173,8cyclictest0-21swapper/109:28:111
359399171,8cyclictest0-21swapper/111:33:011
359399171,13cyclictest9358-21sshd11:45:111
3593991710,5cyclictest22758-21sshd07:27:461
3593991710,5cyclictest19131-21sshd07:56:111
358899178,6cyclictest29347-21sshd11:20:110
358899172,12cyclictest23979-21cron08:05:010
361399161,12cyclictest35966irq/58-eth008:50:123
361399161,10cyclictest0-21swapper/307:40:283
359399162,2cyclictest0-21swapper/111:50:171
359399161,6cyclictest20614-21/usr/sbin/munin08:35:101
359399161,13cyclictest28260-21grep07:35:171
359399160,13cyclictest1381-21apt-get08:15:141
358899166,7cyclictest30421-21df_inode08:45:140
358899164,9cyclictest23726-21kthreadcore07:30:190
358899162,10cyclictest12-21ksoftirqd/008:20:120
358899162,10cyclictest12-21ksoftirqd/008:20:120
359399152,10cyclictest4254-21kthreadcore08:50:201
359399151,11cyclictest9207-21kernelversion07:15:191
361399141,8cyclictest0-21swapper/307:45:143
361399141,11cyclictest0-21swapper/308:10:173
358899142,9cyclictest3688-21latency_hist07:10:000
361399132,9cyclictest0-21swapper/308:45:273
361399132,9cyclictest0-21swapper/307:50:143
361399132,9cyclictest0-21swapper/307:10:173
361399132,6cyclictest0-21swapper/309:05:313
361399131,8cyclictest0-21swapper/307:35:123
361399131,6cyclictest0-21swapper/308:20:263
361399131,6cyclictest0-21swapper/308:20:263
359399131,9cyclictest3580-21cyclictest07:10:011
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional