You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-12 - 15:11

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Sun Apr 12, 2026 12:45:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1705099632,40cyclictest4583-21ssh10:40:120
1706699583,50cyclictest540-21ssh10:20:142
1705699543,8cyclictest131rcu_preempt10:45:131
1705699542,46cyclictest7264-21ssh11:35:161
17056995340,8cyclictest2536-21diskmemload10:55:121
1706699522,45cyclictest0-21swapper/211:15:132
1706699522,2cyclictest22346irq/40-dwc2_hso08:15:122
1706699521,42cyclictest0-21swapper/210:15:122
1705699521,40cyclictest5762-21ssh10:05:121
1706699512,44cyclictest0-21swapper/208:25:122
1706699511,43cyclictest0-21swapper/211:50:142
1705699515,40cyclictest23899-21ssh10:15:121
1706699501,44cyclictest0-21swapper/209:30:122
1706899497,23cyclictest28366irq/58-eth010:35:143
17050994929,8cyclictest12-21ksoftirqd/008:40:120
1706699482,39cyclictest22346irq/40-dwc2_hso08:05:132
1706699482,38cyclictest0-21swapper/210:05:122
17056994818,23cyclictest21-21ksoftirqd/107:35:141
1705099482,38cyclictest30329-21ssh09:25:130
1706699472,21cyclictest851-21sshd09:14:412
1706699471,3cyclictest0-21swapper/208:35:122
1706699462,2cyclictest22010-21apt-get07:15:112
1705099462,34cyclictest14840-21ssh10:10:120
1706699451,6cyclictest29230-21ssh10:00:122
1706699451,3cyclictest0-21swapper/207:55:112
1706699451,2cyclictest29072-21apt-get12:05:122
1706699442,2cyclictest0-21swapper/208:20:142
1706699431,3cyclictest0-21swapper/207:10:132
1706699430,3cyclictest22346irq/40-dwc2_hso12:10:132
1706899422,22cyclictest28366irq/58-eth008:30:143
1706899422,22cyclictest28366irq/58-eth008:30:143
1706699420,3cyclictest20121-21ssh12:00:162
17050994222,9cyclictest12-21ksoftirqd/009:35:130
1706699413,32cyclictest13711-21apt-get10:45:132
1706699411,34cyclictest0-21swapper/209:50:132
1706699411,2cyclictest0-21swapper/209:35:132
1705699416,21cyclictest21-21ksoftirqd/111:30:101
17056994132,4cyclictest2536-21diskmemload09:45:131
1706699401,6cyclictest9748-21cpu10:25:122
17056994015,19cyclictest21-21ksoftirqd/109:05:141
1706699392,7cyclictest22346irq/40-dwc2_hso08:30:142
1706699392,7cyclictest22346irq/40-dwc2_hso08:30:142
1706699392,31cyclictest876-21kthreadcore10:55:192
1706699392,2cyclictest0-21swapper/209:25:122
1706699391,3cyclictest14708-21apt-get12:15:122
1706699391,3cyclictest0-21swapper/209:59:122
1706699391,2cyclictest0-21swapper/208:50:122
1705099391,31cyclictest27852-21ssh10:35:130
1706899382,23cyclictest28366irq/58-eth011:25:113
1706699384,30cyclictest9904-21tr11:35:242
1706699381,6cyclictest0-21swapper/210:40:122
1706699381,31cyclictest11033-21apt-get11:55:132
1706699381,2cyclictest8598-21apt-get11:00:152
1706699380,4cyclictest27846-21apt-get10:35:132
17056993812,18cyclictest201rcuc/108:00:131
1705099381,24cyclictest16322-21ssh11:40:130
1706699371,3cyclictest0-21swapper/210:50:132
1706699371,2cyclictest0-21swapper/209:20:132
1706699370,3cyclictest22346irq/40-dwc2_hso11:30:102
1705699373,26cyclictest2536-21diskmemload09:50:121
17056993722,7cyclictest21-21ksoftirqd/111:05:101
1705699371,28cyclictest0-21swapper/107:45:131
1705099374,4cyclictest131rcu_preempt12:20:120
1705099372,28cyclictest29084-21ssh12:05:120
1706899363,20cyclictest28366irq/58-eth007:10:123
1706699364,22cyclictest0-21swapper/212:30:152
1706699361,7cyclictest10237-21grep08:40:282
1706699361,3cyclictest7454-21/usr/sbin/munin07:30:252
1706699361,3cyclictest0-21swapper/211:05:102
1706699360,4cyclictest1912-21apt-get09:45:132
1706899355,24cyclictest0-21swapper/311:10:143
1706699354,27cyclictest16796-21cat11:20:302
1706699353,25cyclictest0-21swapper/211:29:192
1706699352,4cyclictest0-21swapper/207:20:132
1706699351,7cyclictest15810-21cut07:40:202
1706699351,6cyclictest22002-21mailstats07:45:282
1706699351,2cyclictest18770-21apt-get10:30:132
1706699351,2cyclictest0-21swapper/208:55:212
1706699351,27cyclictest14563-21munin-run08:15:012
1706699350,2cyclictest2536-21diskmemload12:20:122
1705699352,7cyclictest131rcu_preempt07:50:141
1705699352,23cyclictest23715-21apt-get12:20:121
17056993515,10cyclictest17039-21cyclictest09:15:141
1705099356,19cyclictest750-21sendmail-mta11:20:140
1705099354,22cyclictest18534-21ssh12:35:150
1705099353,19cyclictest2910-21ssh09:10:110
1705099351,25cyclictest2536-21diskmemload11:50:120
1706899344,22cyclictest28366irq/58-eth010:25:123
1706899342,17cyclictest28366irq/58-eth008:40:133
1706699342,2cyclictest0-21swapper/208:00:122
1706699342,29cyclictest9850-21kthreadcore07:35:212
1706699341,29cyclictest30248-21ls11:10:242
1706699340,6cyclictest31227-21ssh11:46:432
1706699340,5cyclictest2536-21diskmemload10:12:512
1706699340,5cyclictest15781-21ntp_states09:15:262
1705699341,25cyclictest22818-21ssh10:50:131
17050993421,5cyclictest2536-21diskmemload12:00:170
1706899335,18cyclictest28366irq/58-eth012:25:123
1706899334,23cyclictest28366irq/58-eth009:45:133
1706699331,7cyclictest26659-21ntpq07:50:242
1706699331,6cyclictest3010-21grep07:25:272
1706699331,4cyclictest29051-21ntp_states09:00:262
1706699331,3cyclictest0-21swapper/209:05:292
1705699338,15cyclictest21-21ksoftirqd/111:55:121
1705699331,27cyclictest21193-21ssh09:20:121
1706899321,18cyclictest28366irq/58-eth009:25:123
1706899321,17cyclictest28366irq/58-eth011:15:143
1706699327,22cyclictest21813-21sh11:41:182
1706699322,2cyclictest0-21swapper/209:40:122
1706699320,29cyclictest32696-21ssh12:25:032
1705699323,18cyclictest25254-21ssh09:40:121
1705099322,23cyclictest8630-21ssh11:00:150
1705099322,21cyclictest7036-21ssh09:30:130
17050993210,15cyclictest5726-21ssh12:10:130
1706699311,1cyclictest0-21swapper/212:35:142
17050993115,11cyclictest12-21ksoftirqd/012:15:120
1705099311,16cyclictest18773-21ssh10:30:120
17050993111,15cyclictest21008-1kworker/u9:011:35:140
1706899303,18cyclictest0-21swapper/309:14:413
1706899303,16cyclictest28366irq/58-eth011:05:103
1706899302,13cyclictest28366irq/58-eth009:40:123
1706699302,24cyclictest11159-21/usr/sbin/munin08:45:152
1705699303,17cyclictest21-21ksoftirqd/111:15:131
1705699302,22cyclictest11319-21sshd09:14:411
1706899292,20cyclictest28366irq/58-eth010:30:133
17068992914,9cyclictest28366irq/58-eth009:55:133
1705099292,21cyclictest12728-21cat09:50:190
17068992815,10cyclictest0-21swapper/310:05:133
1706899281,19cyclictest28366irq/58-eth010:45:123
1705099283,19cyclictest21457-21ssh11:25:110
1705099283,19cyclictest12-21ksoftirqd/007:25:120
1705099281,21cyclictest549-21ssh10:20:140
1706899273,16cyclictest0-21swapper/309:50:193
1706899273,15cyclictest0-21swapper/309:20:193
17068992716,6cyclictest0-21swapper/312:05:143
1705699272,20cyclictest26745-21ssh11:10:141
1705699272,19cyclictest29588-21sh11:29:191
1705699272,19cyclictest24676-21ssh09:39:481
1705099273,5cyclictest29220-21apt-get10:00:120
1705099272,18cyclictest2536-21diskmemload11:05:060
1706899263,15cyclictest0-21swapper/310:04:523
17068992617,6cyclictest0-21swapper/310:15:133
17056992610,7cyclictest17039-21cyclictest08:55:141
1706899255,12cyclictest28366irq/58-eth012:00:163
17056992510,3cyclictest21-21ksoftirqd/111:04:121
1705099254,12cyclictest0-21swapper/007:10:130
1706899243,14cyclictest0-21swapper/309:30:193
1706899242,13cyclictest28366irq/58-eth012:10:123
1705699246,11cyclictest201rcuc/110:00:121
1705699242,14cyclictest0-21swapper/109:30:131
17056992414,7cyclictest9072-21cron12:30:011
1705099244,11cyclictest11049-21ssh11:55:140
1705099242,14cyclictest25272-21sh09:40:130
1705099242,13cyclictest9686-21ssh10:25:110
17050992414,7cyclictest13254-21cron10:45:020
1706899233,13cyclictest0-21swapper/309:39:483
1706899232,10cyclictest28366irq/58-eth007:50:133
17068992314,7cyclictest33-21ksoftirqd/311:50:143
17068992313,5cyclictest0-21swapper/310:50:143
1706699230,6cyclictest17084-21latency_hist07:10:022
1705699233,5cyclictest131rcu_preempt08:15:131
1705699232,16cyclictest2536-21diskmemload10:27:191
17056992313,7cyclictest10942-21cron08:45:011
1705699231,17cyclictest5725-21sh12:10:131
1706899221,13cyclictest28366irq/58-eth007:30:113
1705699226,7cyclictest21-21ksoftirqd/110:36:191
1705699222,15cyclictest21-21ksoftirqd/108:05:131
1705699221,16cyclictest18802-21ssh10:30:141
17056992210,3cyclictest21-21ksoftirqd/112:20:011
1705099226,10cyclictest1933-21diskmemload09:45:130
1705099223,16cyclictest12-21ksoftirqd/012:25:110
1705099222,16cyclictest20148-21diskmemload09:55:130
1705099221,17cyclictest21264-21cpuspeed09:20:140
1705099221,15cyclictest32589-21latency_hist10:20:020
17050992210,8cyclictest12-21ksoftirqd/012:30:110
1705099220,7cyclictest482-21kthreadcore11:30:190
1706899213,12cyclictest0-21swapper/312:20:253
17068992112,5cyclictest0-21swapper/312:35:013
1706899211,14cyclictest28366irq/58-eth011:00:133
1705699219,9cyclictest21-21ksoftirqd/112:09:431
1705699219,10cyclictest0-21swapper/111:40:081
17056992114,4cyclictest21-21ksoftirqd/112:35:021
17056992113,5cyclictest0-21swapper/108:20:161
1705699211,14cyclictest20402-21ls09:55:161
17056992110,2cyclictest21-21ksoftirqd/110:23:151
1705099212,14cyclictest12035-21ssh09:15:140
1706899209,8cyclictest0-21swapper/311:55:223
1706899203,10cyclictest0-21swapper/309:19:423
1706899202,13cyclictest0-21swapper/310:40:133
17068992011,6cyclictest28366irq/58-eth011:45:123
17068992010,8cyclictest0-21swapper/311:37:383
1705699209,3cyclictest21-21ksoftirqd/110:15:021
1705699201,9cyclictest334-21wc08:30:291
1705699201,9cyclictest334-21wc08:30:291
17056992014,4cyclictest0-21swapper/112:35:141
17056992014,3cyclictest21-21ksoftirqd/108:45:221
17056992012,5cyclictest0-21swapper/107:25:241
1705699201,15cyclictest24390-21apt-get08:25:121
17056992010,2cyclictest21-21ksoftirqd/111:48:551
1705099201,4cyclictest349-21apt-get08:00:130
1706899199,8cyclictest0-21swapper/311:41:423
1706899193,9cyclictest0-21swapper/310:55:193
1706899193,11cyclictest0-21swapper/311:20:123
1706899193,11cyclictest0-21swapper/310:14:543
1706899192,11cyclictest0-21swapper/307:40:133
1706899192,10cyclictest0-21swapper/308:00:143
1706899192,10cyclictest0-21swapper/307:35:153
1706899191,12cyclictest0-21swapper/307:55:153
17068991910,6cyclictest0-21swapper/310:24:043
1705699198,2cyclictest21-21ksoftirqd/111:50:431
1705699196,4cyclictest21-21ksoftirqd/107:35:011
1705699195,9cyclictest0-21swapper/111:20:121
1705699193,12cyclictest21-21ksoftirqd/112:04:191
1705699192,12cyclictest4933-21/usr/sbin/munin10:40:171
1705699192,11cyclictest1418-21ssh09:25:221
17056991912,4cyclictest0-21swapper/107:40:131
1705699191,15cyclictest26745-21cat07:20:011
1705099195,4cyclictest131rcu_preempt07:30:200
1705099194,4cyclictest846-21kthreadcore10:55:190
1705099191,9cyclictest14120-21irqrtprio07:40:170
1705099191,5cyclictest15758-21munin-run08:50:010
17050991912,4cyclictest12020-21idleruntime-cro11:20:010
1705099191,15cyclictest25328-21ssh11:45:130
17050991910,6cyclictest10092-21cut08:10:160
1705099190,16cyclictest300-21ls08:00:020
1705099190,13cyclictest25693-21timerandwakeup07:15:260
1706899189,6cyclictest0-21swapper/312:35:213
1706899189,6cyclictest0-21swapper/312:15:163
1706899182,10cyclictest0-21swapper/307:20:143
1706899181,13cyclictest0-21swapper/308:25:273
1705699182,2cyclictest131rcu_preempt07:20:171
1705699181,9cyclictest1688-21fschecks_time08:35:161
1705099188,6cyclictest16206-21hwlatdetect08:50:160
1705099185,10cyclictest27639-21cat08:25:220
1705099182,11cyclictest0-21swapper/009:00:150
1705099181,9cyclictest7781-21kthreadcore10:05:200
17050991811,4cyclictest30570-21grep09:05:180
1705099181,14cyclictest29599-21irqcore08:30:190
1705099181,14cyclictest29599-21irqcore08:30:190
17050991811,4cyclictest0-21swapper/011:10:250
1705099181,11cyclictest18512-21timerwakeupswit08:15:270
17050991810,6cyclictest31367-21sshd07:22:180
17050991810,6cyclictest1759-21wc08:35:160
17050991810,5cyclictest24798-21kthreadcore10:50:220
17050991810,5cyclictest0-21swapper/007:45:190
1706899173,8cyclictest0-21swapper/307:45:133
1706899172,10cyclictest0-21swapper/311:30:123
1706899172,10cyclictest0-21swapper/308:45:143
1706899172,10cyclictest0-21swapper/307:25:133
1706899172,10cyclictest0-21swapper/307:15:133
1706899171,13cyclictest0-21swapper/308:20:173
1706899171,12cyclictest0-21swapper/308:15:273
1705699171,7cyclictest30260-21latency_hist09:05:011
17056991712,3cyclictest0-21swapper/108:10:141
1705699171,13cyclictest30367-21kthreadcore07:55:201
1705099173,10cyclictest0-21swapper/008:20:130
1706899162,10cyclictest0-21swapper/309:05:153
1706899162,10cyclictest0-21swapper/308:05:143
1705699161,12cyclictest16284-21if_eth108:50:171
1705699161,11cyclictest21083-21unixbench_singl07:10:281
1705099168,6cyclictest5305-21awk08:05:160
1705099167,5cyclictest9697-21kthreadcore07:35:200
1706899152,9cyclictest0-21swapper/308:10:143
1705699151,11cyclictest16931-21apt-get07:10:021
1705099151,8cyclictest23409-21cut07:50:160
1705099151,12cyclictest17116-21cmp07:10:020
1705099150,2cyclictest131rcu_preempt08:55:130
1706899143,9cyclictest0-21swapper/308:35:143
1706899131,9cyclictest0-21swapper/307:10:023
1706899131,7cyclictest0-21swapper/308:55:283
1706899131,5cyclictest0-21swapper/308:50:153
1706899121,9cyclictest0-21swapper/309:00:183
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional