You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-21 - 06:07

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Fri Aug 21, 2026 00:45:10)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
26668997117,6cyclictest131rcu_preempt20:35:122
2666899639,14cyclictest131rcu_preempt20:25:112
26680995928,23cyclictest35966irq/58-eth023:45:153
2664899554,18cyclictest12-21ksoftirqd/023:55:130
2664899554,18cyclictest12-21ksoftirqd/023:55:130
2666099523,40cyclictest15352-21ssh23:35:151
26648995119,9cyclictest131rcu_preempt21:50:140
2666099492,5cyclictest131rcu_preempt00:20:131
26668994615,12cyclictest0-21swapper/223:45:142
2664899463,36cyclictest12850-21ssh21:10:130
2664899454,36cyclictest29321-21ssh21:55:120
2664899446,35cyclictest25612-21ssh21:35:140
2668099424,23cyclictest35966irq/58-eth021:25:153
2668099422,27cyclictest35966irq/58-eth022:50:123
2664899425,27cyclictest14991-21ssh22:05:140
26680994020,13cyclictest35966irq/58-eth023:00:133
2666099405,7cyclictest21-21ksoftirqd/121:10:111
26660994025,7cyclictest0-21swapper/123:50:141
2668099393,25cyclictest35966irq/58-eth020:20:133
26668993820,12cyclictest27-21ksoftirqd/221:10:132
2666099381,4cyclictest131rcu_preempt19:55:141
26648993822,12cyclictest12-21ksoftirqd/022:55:130
2666899372,19cyclictest10747-21ssh21:44:562
26660993727,6cyclictest8548-21kworker/1:219:25:141
2668099363,22cyclictest35966irq/58-eth020:30:133
2668099362,17cyclictest35966irq/58-eth000:25:133
2666899362,28cyclictest2600-21sh00:39:562
2664899368,21cyclictest12-21ksoftirqd/022:35:120
2668099353,20cyclictest35966irq/58-eth021:30:143
26680993510,18cyclictest35966irq/58-eth023:05:143
2666899358,22cyclictest31445-21ssh22:50:142
2664899351,29cyclictest24338-21ssh23:40:120
26648993510,20cyclictest12-21ksoftirqd/019:10:120
2668099345,20cyclictest35966irq/58-eth023:25:123
2668099342,19cyclictest35966irq/58-eth000:20:133
26668993411,15cyclictest0-21swapper/219:55:122
2666099340,25cyclictest17453-21ssh00:30:151
2664899348,18cyclictest12-21ksoftirqd/000:05:160
2664899344,19cyclictest27982-21ssh00:00:130
2664899344,19cyclictest27982-21ssh00:00:120
2664899343,18cyclictest12-21ksoftirqd/020:35:130
2668099332,18cyclictest35966irq/58-eth020:40:123
26680993316,11cyclictest0-21swapper/321:36:563
26668993322,7cyclictest30951-21ssh21:20:132
2666099338,17cyclictest21-21ksoftirqd/100:05:151
26660993315,11cyclictest21-21ksoftirqd/120:10:121
2666099331,27cyclictest0-21swapper/119:20:101
2668099324,18cyclictest35966irq/58-eth023:20:123
2668099321,16cyclictest35966irq/58-eth020:00:133
2666899324,8cyclictest0-21swapper/220:10:122
2664899322,23cyclictest8401-21ssh00:25:140
26648993214,5cyclictest131rcu_preempt23:20:110
26648993213,12cyclictest12-21ksoftirqd/023:05:120
2668099312,16cyclictest35966irq/58-eth023:10:133
2668099311,16cyclictest35966irq/58-eth022:10:133
2666899312,23cyclictest420-21ssh23:44:572
2664899318,7cyclictest12-21ksoftirqd/000:20:130
2664899312,26cyclictest5968-21ssh22:00:150
26680993021,5cyclictest0-21swapper/300:00:133
26680993021,5cyclictest0-21swapper/300:00:133
2664899304,22cyclictest12-21ksoftirqd/022:20:140
2664899304,17cyclictest16577-21ssh21:30:130
2664899303,20cyclictest556-21ssh22:15:130
2664899303,19cyclictest844-21munin-node23:25:120
2664899301,23cyclictest13368-21ssh22:40:130
2666099292,20cyclictest26430-21rm00:35:141
2666099291,3cyclictest0-21swapper/119:50:141
2668099281,17cyclictest35966irq/58-eth021:05:133
2666899282,11cyclictest25937-21ssh00:34:562
26668992812,12cyclictest26631-21cyclictest23:55:142
26668992812,12cyclictest26631-21cyclictest23:55:142
2666099282,18cyclictest17086-21ssh23:00:141
26660992817,6cyclictest0-21swapper/123:30:141
26660992816,9cyclictest48-21kcompactd021:40:121
26648992812,12cyclictest2662-21cron00:40:010
2668099273,15cyclictest35966irq/58-eth023:30:133
2666899274,16cyclictest27346irq/40-dwc2_hso22:45:052
2666899274,16cyclictest27346irq/40-dwc2_hso22:45:042
2664899279,12cyclictest12-21ksoftirqd/020:25:120
2664899271,19cyclictest18685-21ssh22:25:140
2666899261,9cyclictest16104-21ssh21:29:562
2666899261,12cyclictest4076-21ls19:20:132
2666899260,14cyclictest23229-21find19:40:102
2666099261,18cyclictest26166-21ssh23:05:141
2664899261,18cyclictest12-21ksoftirqd/000:10:120
2668099256,13cyclictest35966irq/58-eth022:20:153
2666899254,15cyclictest27346irq/40-dwc2_hso23:30:122
2666899252,15cyclictest2787-21ssh23:10:132
2666899251,16cyclictest27346irq/40-dwc2_hso19:30:122
2666099254,17cyclictest201rcuc/123:45:141
2666099252,18cyclictest12570-21ssh22:02:391
2666099250,18cyclictest27711-21ssh22:30:141
2664899254,13cyclictest17082-21ssh23:00:140
2664899253,15cyclictest0-21swapper/020:40:130
26648992511,9cyclictest26631-21cyclictest22:30:130
2668099244,17cyclictest35966irq/58-eth023:15:143
2668099243,13cyclictest0-21swapper/321:42:323
2666899244,10cyclictest32520-21diskmemload22:30:322
2666899242,16cyclictest27346irq/40-dwc2_hso23:35:152
2666899242,14cyclictest8054-21ssh22:55:132
2666899241,15cyclictest156-21systemd-udevd22:08:472
2666099241,7cyclictest12452-21munin-plugin-st21:10:011
2664899242,19cyclictest7408-21ssh21:40:560
2668099237,9cyclictest35966irq/58-eth000:15:153
2666899232,6cyclictest0-21swapper/220:30:142
2666899232,6cyclictest0-21swapper/200:07:542
2666899232,16cyclictest0-21swapper/221:45:562
26668992311,9cyclictest31651-21ssh00:20:062
26668992311,9cyclictest21795-21usage_EthcTask23:00:302
2666899231,16cyclictest27346irq/40-dwc2_hso20:20:122
2666099231,8cyclictest30732-21cut00:00:221
2666099231,8cyclictest30732-21cut00:00:221
2664899239,8cyclictest12-21ksoftirqd/023:15:150
2664899236,13cyclictest0-21swapper/000:30:140
2664899231,19cyclictest12473-21ls21:10:010
26648992311,7cyclictest13094-21kworker/u8:123:54:560
26648992310,10cyclictest24322-21cron20:15:020
26680992211,5cyclictest0-21swapper/323:50:143
2666899227,4cyclictest0-21swapper/223:05:182
2666899227,11cyclictest21974-21forks21:15:162
2666899223,14cyclictest27346irq/40-dwc2_hso19:15:142
2666899222,16cyclictest20965-21kthreadcore22:25:222
2666899222,15cyclictest0-21swapper/221:38:572
26668992211,9cyclictest22934-21ssh21:32:202
26668992211,8cyclictest32236-21kthreadcore21:55:212
2666899221,12cyclictest17142-21awk22:40:262
2666899221,11cyclictest4288-21ntp_states22:15:252
26668992210,10cyclictest12607-21ssh22:39:422
26660992215,3cyclictest21-21ksoftirqd/122:40:131
2666099221,13cyclictest26048-21cut20:50:161
2664899228,11cyclictest31028-21cron22:50:010
2664899228,11cyclictest31028-21cron22:50:010
2664899226,11cyclictest12-21ksoftirqd/021:27:550
26648992214,5cyclictest6522-21ssh23:30:170
2668099219,8cyclictest35966irq/58-eth021:50:153
2668099212,15cyclictest35966irq/58-eth000:10:123
2666899219,9cyclictest20039-21ssh00:12:202
2666899219,9cyclictest11730-21ssh00:25:242
2666899219,5cyclictest3488-21ssh23:27:112
2666899218,10cyclictest26064-21fschecks_time20:50:162
2666899213,15cyclictest8957-21grep19:25:172
26668992110,9cyclictest30957-21ssh22:13:012
26668992110,9cyclictest28693-21ssh23:24:142
26668992110,9cyclictest24463-21/usr/sbin/munin20:45:242
26668992110,9cyclictest24463-21/usr/sbin/munin20:45:242
2666899210,11cyclictest32694-21cat19:50:022
2666099217,8cyclictest0-21swapper/120:20:121
2666099215,4cyclictest131rcu_preempt21:50:011
26660992112,6cyclictest24431-21grep21:50:311
2666099211,15cyclictest26661-21ssh00:15:311
2666099211,13cyclictest10300-21ls20:00:161
2664899217,4cyclictest12-21ksoftirqd/019:29:550
2664899217,10cyclictest12-21ksoftirqd/023:10:130
2664899211,14cyclictest24082-21ssh22:10:140
2668099209,8cyclictest0-21swapper/322:31:573
26680992010,8cyclictest0-21swapper/322:46:233
26680992010,8cyclictest0-21swapper/322:46:233
26680992010,6cyclictest0-21swapper/322:15:283
2666899208,4cyclictest27961-21sh00:00:122
2666899208,4cyclictest27961-21sh00:00:122
2666899206,11cyclictest6923-21grep21:00:292
2666899203,9cyclictest23265-21irqcore00:15:202
2666899202,10cyclictest23943-21ntp_192.168.11421:50:262
2666899201,9cyclictest20337-21sendmail23:20:002
2666899201,9cyclictest0-21swapper/219:35:102
2666899201,8cyclictest11764-21kthreadcore20:00:202
2666899201,15cyclictest13298-21ntp_192.168.11422:20:262
2666899201,11cyclictest19491-21wc20:40:232
2666899201,10cyclictest4215-21grep19:50:292
2666899201,10cyclictest18347-21ntp_states20:05:272
2666099203,3cyclictest131rcu_preempt21:15:191
2666099201,9cyclictest0-21swapper/100:25:311
2666099201,8cyclictest3781-21munin-run19:20:021
2666099201,5cyclictest2809-21apt21:00:011
26660992014,4cyclictest0-21swapper/120:15:161
26660992012,6cyclictest0-21swapper/123:20:131
26660992012,5cyclictest0-21swapper/119:30:221
26660992010,7cyclictest25645-21cut19:40:201
2666099200,2cyclictest131rcu_preempt22:10:141
2666099200,18cyclictest22184-21ls19:35:261
2664899208,3cyclictest12-21ksoftirqd/021:48:440
2664899207,3cyclictest12-21ksoftirqd/020:00:020
2664899201,9cyclictest7798-21latency_hist21:05:020
26648992012,6cyclictest26618-21cat21:15:320
26648992011,6cyclictest23940-21cron23:40:010
2664899200,9cyclictest12-21ksoftirqd/020:05:140
2668099199,8cyclictest0-21swapper/323:55:243
2668099199,8cyclictest0-21swapper/323:55:233
2668099197,9cyclictest0-21swapper/321:10:303
2668099193,11cyclictest0-21swapper/300:37:563
2668099192,11cyclictest0-21swapper/300:32:563
2668099192,10cyclictest0-21swapper/320:25:133
26680991910,6cyclictest35966irq/58-eth022:25:143
26680991910,6cyclictest35966irq/58-eth021:55:133
26680991910,6cyclictest0-21swapper/322:35:313
2666899195,6cyclictest0-21swapper/220:15:122
2666899193,9cyclictest10805-21kthreadcore21:05:222
2666899192,15cyclictest27341-21irqrtprio19:10:172
2666899191,9cyclictest30972-21ps20:55:172
2666899191,8cyclictest8489-21kthreadcore22:00:232
2666899191,11cyclictest9951-21ssh23:50:152
2666099197,3cyclictest29832-21sh23:41:151
2666099196,3cyclictest12446-21diskmemload21:26:471
2666099192,9cyclictest8025-21ssh22:55:121
2666099192,11cyclictest31418-21apt-get22:50:141
2666099191,8cyclictest6215-21sed23:10:231
2666099191,4cyclictest29300-21apt-get21:55:121
26660991912,4cyclictest0-21swapper/121:00:161
26660991911,6cyclictest6211-21sshd21:23:571
2666099191,16cyclictest20765-21apt-key20:45:021
2666099191,15cyclictest25595-21apt-get21:35:131
2666099191,12cyclictest6776-21uniq20:30:171
2664899199,7cyclictest31051-21df_inode21:20:140
2664899193,6cyclictest12-21ksoftirqd/020:50:010
2664899193,6cyclictest12-21ksoftirqd/020:50:000
2664899193,13cyclictest22783-21ssh00:15:140
2664899191,13cyclictest31498-21cpuspeed22:50:140
2668099188,8cyclictest0-21swapper/300:05:593
2668099188,6cyclictest35966irq/58-eth021:45:133
2668099183,10cyclictest0-21swapper/322:04:563
2668099182,10cyclictest0-21swapper/320:45:053
2668099182,10cyclictest0-21swapper/320:45:053
2668099182,10cyclictest0-21swapper/320:35:133
2668099182,10cyclictest0-21swapper/319:25:133
2668099182,10cyclictest0-21swapper/319:10:133
26680991811,4cyclictest0-21swapper/321:15:153
2668099181,11cyclictest0-21swapper/319:50:013
26680991810,6cyclictest0-21swapper/323:40:023
2666099189,7cyclictest4507-21perl22:35:151
2666099189,7cyclictest27304-21cron22:30:011
2666099186,9cyclictest25189-21cron21:35:001
2666099182,3cyclictest131rcu_preempt22:15:201
2666099181,8cyclictest965-21ntp_states23:25:261
2666099181,5cyclictest13591-21/usr/sbin/munin00:10:151
26660991812,4cyclictest0-21swapper/122:05:181
2666099181,11cyclictest22502-21ssh22:45:151
2666099181,11cyclictest22502-21ssh22:45:151
26660991810,5cyclictest21-21ksoftirqd/122:20:191
2666099180,4cyclictest22768-21awk23:55:291
2666099180,4cyclictest22768-21awk23:55:291
2664899189,2cyclictest12-21ksoftirqd/020:55:140
2664899188,6cyclictest614-21dbus-daemon19:22:060
2664899187,2cyclictest12-21ksoftirqd/020:20:250
2664899186,9cyclictest24887-21tail20:15:160
2664899186,2cyclictest12-21ksoftirqd/019:47:110
26648991812,3cyclictest12-21ksoftirqd/019:20:020
2664899181,13cyclictest4624-21ntp_states23:45:280
2668099179,6cyclictest0-21swapper/323:36:143
2668099179,6cyclictest0-21swapper/322:07:073
2668099179,5cyclictest35966irq/58-eth022:55:123
2668099179,3cyclictest0-21swapper/322:40:533
2668099172,10cyclictest0-21swapper/321:20:133
2668099172,10cyclictest0-21swapper/320:15:123
2668099171,9cyclictest0-21swapper/319:35:093
2668099171,13cyclictest0-21swapper/320:10:273
2668099171,10cyclictest0-21swapper/319:15:283
2666899171,8cyclictest26570-21latency_hist19:10:012
2666099177,7cyclictest13563-21cut20:35:201
2666099173,11cyclictest21-21ksoftirqd/123:15:151
2666099171,8cyclictest29224-21tr19:10:191
26660991711,4cyclictest0-21swapper/120:45:161
26660991711,4cyclictest0-21swapper/120:45:161
26660991711,4cyclictest0-21swapper/119:45:141
2664899177,8cyclictest10070-21apt-get20:00:120
2664899173,11cyclictest27796-21apt19:45:020
2664899172,10cyclictest0-21swapper/020:50:130
2664899171,14cyclictest18202-21latency_hist19:35:010
2664899171,12cyclictest454-21diskstats19:50:150
2668099161,12cyclictest0-21swapper/319:20:153
2666099162,9cyclictest0-21swapper/120:05:151
2666099161,12cyclictest2080-21sort20:25:171
2664899162,11cyclictest20069-21kthreadcore19:35:180
2664899161,12cyclictest6592-21apt-get20:30:130
2668099143,8cyclictest0-21swapper/319:40:213
2668099142,10cyclictest0-21swapper/320:05:153
2668099141,7cyclictest0-21swapper/321:05:023
2668099141,7cyclictest0-21swapper/320:55:273
2668099141,7cyclictest0-21swapper/320:50:143
2668099141,6cyclictest0-21swapper/319:55:173
2664899142,9cyclictest26650-21latency_hist19:10:010
2668099138,3cyclictest0-21swapper/319:30:243
2668099131,5cyclictest0-21swapper/319:50:203
2666099131,9cyclictest26748-21latency_hist19:10:011
2668099111,7cyclictest0-21swapper/319:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional