You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-11 - 13:40

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Wed Mar 11, 2026 00:45:11)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
845799594,48cyclictest25645-21ssh22:40:111
8446995031,7cyclictest12-21ksoftirqd/022:25:110
8475994933,13cyclictest28366irq/58-eth022:00:123
8475994933,13cyclictest28366irq/58-eth022:00:123
8446994941,3cyclictest26820-21diskmemload22:10:120
8475994829,9cyclictest28366irq/58-eth000:20:133
8475994820,17cyclictest28366irq/58-eth021:35:143
846699481,21cyclictest4718-21ssh00:35:132
845799481,33cyclictest17871-21ssh23:30:131
8446994838,4cyclictest8969-21ssh23:25:120
844699483,4cyclictest131rcu_preempt22:50:130
844699473,39cyclictest1223-21ssh00:15:140
846699438,26cyclictest633-21Async12:11:532
845799421,34cyclictest26794-21ssh23:35:131
846699412,21cyclictest11217-21ssh21:55:192
8466994120,7cyclictest131rcu_preempt20:30:132
847599402,20cyclictest28366irq/58-eth021:25:123
844699402,25cyclictest7704-21ssh22:30:110
845799392,27cyclictest19242-21ssh00:25:131
845799392,26cyclictest28189-21ssh00:30:121
8475993822,10cyclictest28366irq/58-eth021:10:133
8475993822,10cyclictest28366irq/58-eth021:10:133
847599381,20cyclictest28366irq/58-eth022:30:123
8475993721,11cyclictest33-21ksoftirqd/322:25:123
8466993714,18cyclictest27-21ksoftirqd/222:50:132
847599364,12cyclictest0-21swapper/322:55:153
845799362,26cyclictest20469-21tr22:00:191
845799362,26cyclictest20469-21tr22:00:191
8446993625,5cyclictest0-21swapper/023:55:130
8446993621,7cyclictest0-21swapper/019:25:120
846699352,18cyclictest6857-21ssh21:35:142
845799352,7cyclictest26820-21diskmemload23:10:131
844699354,24cyclictest10042-21ssh21:55:140
844699344,19cyclictest12-21ksoftirqd/021:35:140
847599322,19cyclictest28366irq/58-eth000:15:133
846699326,20cyclictest27-21ksoftirqd/222:25:122
846699325,16cyclictest22346irq/40-dwc2_hso19:25:122
845799322,23cyclictest23502-21ssh23:15:121
845799321,21cyclictest30228-21ssh23:55:131
844699320,23cyclictest5581-21ssh23:05:140
847599312,19cyclictest28366irq/58-eth019:40:123
847599312,17cyclictest0-21swapper/300:30:123
844699311,21cyclictest16727-21ssh22:35:130
846699304,15cyclictest1207-21sh00:15:132
846699303,18cyclictest1149-21ssh21:50:132
846699295,17cyclictest22346irq/40-dwc2_hso20:35:142
846699292,21cyclictest18267-21ssh00:24:202
845799298,10cyclictest21545-21ssh21:25:121
845799295,11cyclictest21-21ksoftirqd/122:50:151
846699286,16cyclictest0-21swapper/220:00:132
846699283,13cyclictest27881-21ssh22:05:122
846699282,20cyclictest0-21swapper/220:45:132
846699281,20cyclictest73-21kswapd022:35:042
844699284,18cyclictest4739-21ssh00:35:150
8446992812,8cyclictest3527-21diskstats23:40:150
844699281,19cyclictest24662-21ssh21:45:110
844699281,19cyclictest21295-21ssh23:50:130
847599272,18cyclictest0-21swapper/321:59:203
8475992715,7cyclictest28366irq/58-eth022:50:273
847599271,15cyclictest28366irq/58-eth023:30:113
845799272,18cyclictest29507-21ssh00:10:321
8475992614,8cyclictest28366irq/58-eth023:15:123
8475992611,10cyclictest28366irq/58-eth022:35:133
846699265,13cyclictest22346irq/40-dwc2_hso00:10:132
846699264,10cyclictest0-21swapper/219:30:112
846699262,10cyclictest0-21swapper/220:10:142
846699261,22cyclictest0-21swapper/223:25:132
845799267,4cyclictest131rcu_preempt21:18:261
845799263,17cyclictest28967-21ssh23:00:111
845799262,18cyclictest3416-21ssh23:40:151
844699262,7cyclictest131rcu_preempt19:15:120
844699262,17cyclictest10241-21ssh00:20:120
844699261,19cyclictest8244-21ssh21:53:190
846699252,8cyclictest0-21swapper/222:10:122
846699252,18cyclictest560966ssh23:05:142
846699251,11cyclictest869466ssh23:25:022
8466992511,11cyclictest0-21swapper/222:00:112
8466992511,11cyclictest0-21swapper/222:00:112
8457992513,10cyclictest0-21swapper/119:10:241
845799251,19cyclictest22232-21ssh22:20:131
8457992510,6cyclictest131rcu_preempt21:55:281
844699252,17cyclictest26820-21diskmemload22:06:190
844699251,10cyclictest16454-21ssh23:10:200
8475992410,8cyclictest28366irq/58-eth023:35:133
846699249,12cyclictest23666-21fschecks_count23:15:162
846699243,13cyclictest22346irq/40-dwc2_hso22:15:132
846699242,19cyclictest30775-21sort23:35:292
8466992411,6cyclictest1253-21ssh21:12:312
8466992411,6cyclictest1253-21ssh21:12:312
8466992411,6cyclictest10311-21ssh23:43:032
845799249,7cyclictest21-21ksoftirqd/100:04:171
845799243,4cyclictest21-21ksoftirqd/121:40:301
8457992413,3cyclictest0-21swapper/119:55:211
8457992412,7cyclictest201rcuc/121:30:121
845799241,18cyclictest32467-21ssh23:20:141
844699246,6cyclictest0-21swapper/022:15:230
847599237,11cyclictest0-21swapper/300:03:203
847599231,12cyclictest28366irq/58-eth023:45:123
846699239,11cyclictest27924-21kthreadcore22:40:202
846699239,11cyclictest12621-21apt-get20:55:132
846699232,15cyclictest0-21swapper/222:30:052
846699232,13cyclictest2158-21rm22:45:122
8466992311,9cyclictest31981-21kthreadcore23:00:212
846699231,12cyclictest16021-21ntp_states23:45:262
846699231,11cyclictest25316-21grep23:50:282
845799236,4cyclictest131rcu_preempt23:50:131
845799235,10cyclictest21-21ksoftirqd/100:36:151
845799233,15cyclictest7718-21ssh22:30:121
8457992310,11cyclictest0-21swapper/123:49:461
8457992310,11cyclictest0-21swapper/122:36:111
8457992310,11cyclictest0-21swapper/122:09:481
8457992310,11cyclictest0-21swapper/100:21:151
844699231,17cyclictest27230-21ssh21:10:140
844699231,17cyclictest27230-21ssh21:10:140
844699230,17cyclictest23536-21ssh23:15:150
847599222,13cyclictest28366irq/58-eth019:50:143
847599221,18cyclictest0-21swapper/322:10:163
847599221,16cyclictest28366irq/58-eth023:40:133
847599221,14cyclictest28366irq/58-eth021:40:123
8475992210,9cyclictest0-21swapper/321:51:273
846699229,9cyclictest15642-21ssh00:05:142
846699228,6cyclictest11411-21cat00:00:322
846699227,12cyclictest2791-21cron20:45:012
846699224,13cyclictest22346irq/40-dwc2_hso19:45:132
846699223,16cyclictest0-21swapper/223:30:112
846699221,8cyclictest2232766ssh22:20:142
846699221,16cyclictest31934-21processes00:30:272
845799227,6cyclictest21-21ksoftirqd/100:16:471
8457992212,2cyclictest1257-21ssh21:12:321
8457992212,2cyclictest1257-21ssh21:12:321
8457992210,10cyclictest0-21swapper/122:46:521
844699225,10cyclictest8431-21cyclictest21:15:140
844699221,3cyclictest131rcu_preempt22:55:140
8446992212,8cyclictest14983-21sshd21:20:200
844699221,10cyclictest15778-21ssh21:40:140
847599219,9cyclictest0-21swapper/323:21:123
847599213,13cyclictest0-21swapper/323:50:133
847599211,15cyclictest0-21swapper/322:20:133
8475992111,4cyclictest0-21swapper/323:10:133
847599211,10cyclictest28366irq/58-eth020:30:133
846699218,5cyclictest2405-21tr19:35:222
846699218,4cyclictest20292-21cut00:25:192
846699217,11cyclictest30761-21irqcore23:55:192
846699212,16cyclictest13202-21latency_hist19:15:012
846699212,11cyclictest9971-21ntp_states20:15:272
846699211,8cyclictest1869166ssh21:40:232
846699211,8cyclictest17431-21/usr/sbin/munin21:00:122
846699211,8cyclictest17431-21/usr/sbin/munin21:00:122
846699211,17cyclictest14316-21kthreadcore21:20:192
846699211,16cyclictest22346irq/40-dwc2_hso19:15:122
846699211,15cyclictest3374-21sshd21:31:122
846699211,11cyclictest26813-21cron21:10:002
846699211,11cyclictest26813-21cron21:10:002
846699211,11cyclictest26466-21cut21:45:192
846699211,10cyclictest14707-21ssh23:10:152
845799219,3cyclictest21-21ksoftirqd/100:07:581
845799218,11cyclictest0-21swapper/122:28:551
845799218,11cyclictest0-21swapper/121:52:291
845799217,3cyclictest131rcu_preempt21:35:131
845799211,5cyclictest531-21usb-storage23:05:041
8457992115,4cyclictest0-21swapper/121:45:111
8457992110,9cyclictest0-21swapper/123:27:281
8457992110,9cyclictest0-21swapper/122:10:591
844699211,11cyclictest25156-21ntp_192.168.11421:25:240
847599209,8cyclictest0-21swapper/323:25:243
847599209,6cyclictest0-21swapper/300:26:463
847599203,11cyclictest0-21swapper/321:15:153
8475992011,6cyclictest28366irq/58-eth000:10:123
8475992011,4cyclictest0-21swapper/322:41:393
8475992010,8cyclictest0-21swapper/321:46:143
846699203,9cyclictest15116-21irqcore19:50:182
846699201,9cyclictest10050-21tr20:50:212
846699201,16cyclictest16632-21tr20:25:162
846699201,15cyclictest8726-21find19:40:272
846699201,11cyclictest22072-21cut21:25:172
846699201,11cyclictest19697-21fschecks_count19:55:152
846699201,10cyclictest20734-21irqstats22:55:202
845799208,4cyclictest21-21ksoftirqd/120:10:261
845799204,4cyclictest21-21ksoftirqd/119:30:011
845799203,6cyclictest131rcu_preempt20:15:291
845799202,3cyclictest131rcu_preempt20:25:221
845799202,14cyclictest12605-21sh21:20:111
845799201,16cyclictest20049-21ssh22:55:151
8457992010,7cyclictest0-21swapper/120:35:191
845799200,15cyclictest25527-21ls21:05:221
845799200,15cyclictest25527-21ls21:05:221
844699209,8cyclictest11323-21apt-get20:20:130
844699202,11cyclictest0-21swapper/020:45:130
844699201,5cyclictest0-21swapper/023:20:090
8446992012,5cyclictest0-21swapper/000:10:270
844699201,16cyclictest28961-21apt-get23:00:110
847599199,7cyclictest0-21swapper/321:20:263
847599199,3cyclictest0-21swapper/300:06:003
847599193,11cyclictest0-21swapper/322:07:193
847599192,11cyclictest0-21swapper/320:55:143
847599191,12cyclictest0-21swapper/319:15:153
8475991910,6cyclictest28366irq/58-eth021:30:133
8475991910,6cyclictest0-21swapper/300:35:203
846699192,9cyclictest18898-21kthreadcore19:20:202
846699191,15cyclictest29955-21kthreadcore20:05:192
846699191,14cyclictest8561-21latency_hist19:10:022
845799194,13cyclictest26820-21diskmemload22:15:191
845799191,3cyclictest0-21swapper/120:50:231
8457991913,4cyclictest0-21swapper/120:55:191
8457991913,4cyclictest0-21swapper/119:45:261
8457991913,4cyclictest0-21swapper/119:20:151
845799191,10cyclictest32296-21kthreadcore20:05:221
844699198,3cyclictest12-21ksoftirqd/022:24:360
844699192,8cyclictest0-21swapper/000:00:200
844699191,9cyclictest21670-21tr00:25:210
8446991913,4cyclictest0-21swapper/000:05:150
844699191,13cyclictest26820-21diskmemload23:45:130
844699190,7cyclictest131rcu_preempt23:30:070
844699190,14cyclictest2945-21ssh00:33:200
847599189,6cyclictest0-21swapper/323:58:153
847599182,10cyclictest0-21swapper/321:05:133
847599182,10cyclictest0-21swapper/321:05:133
847599182,10cyclictest0-21swapper/320:15:053
847599182,10cyclictest0-21swapper/320:00:053
847599182,10cyclictest0-21swapper/319:55:133
847599182,10cyclictest0-21swapper/319:30:133
847599182,10cyclictest0-21swapper/319:25:133
847599181,12cyclictest0-21swapper/323:10:023
847599181,12cyclictest0-21swapper/322:45:163
847599181,11cyclictest0-21swapper/319:35:293
847599181,10cyclictest0-21swapper/320:40:053
846699181,9cyclictest11313-21acpi20:20:122
845799188,4cyclictest0-21swapper/120:30:131
845799184,3cyclictest131rcu_preempt20:00:271
845799182,11cyclictest0-21swapper/119:50:141
8457991813,3cyclictest0-21swapper/119:15:131
8457991812,4cyclictest0-21swapper/119:30:281
844699189,7cyclictest11916-21memory19:10:230
844699189,7cyclictest11067-21cron20:20:010
844699185,4cyclictest131rcu_preempt21:00:010
844699183,5cyclictest27503-21cut23:35:190
844699181,9cyclictest7565-21tr19:40:210
844699181,4cyclictest0-21swapper/022:46:390
844699181,4cyclictest0-21swapper/021:32:190
844699181,3cyclictest131rcu_preempt22:00:200
844699181,3cyclictest131rcu_preempt22:00:190
8446991812,4cyclictest0-21swapper/020:00:290
844699181,14cyclictest30287-21tr19:30:200
844699181,13cyclictest2220-21kthreadcore19:35:210
844699181,11cyclictest0-21swapper/020:10:130
8446991810,5cyclictest0-21swapper/020:40:020
844699180,15cyclictest22168-21ls21:05:010
844699180,15cyclictest22168-21ls21:05:010
844699180,13cyclictest19341-21latency_hist19:55:010
847599179,3cyclictest0-21swapper/323:01:083
847599172,9cyclictest0-21swapper/320:50:153
847599172,9cyclictest0-21swapper/320:45:133
847599172,10cyclictest0-21swapper/321:00:133
847599172,10cyclictest0-21swapper/321:00:123
847599172,10cyclictest0-21swapper/320:20:143
847599171,12cyclictest0-21swapper/322:15:213
845799173,11cyclictest2046-21kthreadcore19:35:221
8457991712,3cyclictest0-21swapper/120:45:131
8457991711,4cyclictest0-21swapper/119:40:141
845799171,11cyclictest11308-21find20:20:121
844699179,6cyclictest1-21systemd20:25:560
844699178,7cyclictest26755-21cut22:40:180
844699173,6cyclictest21257-21grep20:30:180
844699171,13cyclictest29267-21apt-get20:05:130
844699171,12cyclictest7804-21apt-get20:50:130
844699171,12cyclictest0-21swapper/019:45:130
847599161,12cyclictest0-21swapper/320:10:223
847599161,10cyclictest0-21swapper/319:45:133
845799166,8cyclictest30977-21irqcore20:40:181
845799162,3cyclictest131rcu_preempt19:10:031
8457991610,4cyclictest0-21swapper/121:00:121
8457991610,4cyclictest0-21swapper/121:00:121
844699161,13cyclictest20085-21kthreadcore19:20:230
844699161,11cyclictest24188-21tr21:05:200
844699161,11cyclictest24188-21tr21:05:200
847599151,7cyclictest0-21swapper/319:20:123
844699151,11cyclictest32412-21kthreadcore20:40:210
844699151,11cyclictest19572-21apt-get19:55:130
847599142,10cyclictest0-21swapper/320:05:143
844699141,10cyclictest8448-21latency_hist19:10:020
847599138,3cyclictest0-21swapper/319:10:243
847599132,7cyclictest0-21swapper/320:25:113
847599122,6cyclictest0-21swapper/320:35:133
847599101,6cyclictest0-21swapper/319:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional