You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-16 - 05:28

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot4s.osadl.org (updated Sun Aug 16, 2026 00:45:09)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
148199542,39cyclictest32353-21ssh23:05:151
148199541,25cyclictest19648-21sh00:10:151
151599533,36cyclictest8917-21ssh23:10:142
151599533,36cyclictest8917-21ssh23:10:132
148199521,37cyclictest9224-21ssh21:40:111
1530994940,5cyclictest35966irq/58-eth021:55:133
148199481,14cyclictest0-21swapper/120:35:141
1515994629,8cyclictest27-21ksoftirqd/223:20:132
148199451,37cyclictest14630-21ssh21:25:131
151599437,30cyclictest131rcu_preempt22:20:142
151599437,30cyclictest131rcu_preempt22:20:132
148199425,9cyclictest21-21ksoftirqd/122:20:141
148199425,9cyclictest21-21ksoftirqd/122:20:131
148199424,5cyclictest131rcu_preempt19:25:121
148199421,35cyclictest10650-21ssh00:05:161
1515994113,11cyclictest1436-21cyclictest22:00:162
148199411,28cyclictest3366-21ssh23:25:121
148199401,11cyclictest21-21ksoftirqd/123:40:141
148199391,29cyclictest14401-21ssh22:55:141
148199390,33cyclictest16019-21ssh23:50:131
153099381,23cyclictest0-21swapper/321:10:133
148199373,27cyclictest12386-21ssh23:30:131
1481993716,17cyclictest201rcuc/123:15:141
151599364,24cyclictest19913-21ssh22:40:122
153099353,20cyclictest35966irq/58-eth022:35:133
1530993517,14cyclictest0-21swapper/321:20:143
151599353,3cyclictest0-21swapper/223:35:132
1515993513,7cyclictest1436-21cyclictest22:50:132
151599332,20cyclictest18236-21ssh21:45:122
1515993318,10cyclictest27-21ksoftirqd/222:35:152
148199331,22cyclictest18250-21ssh21:45:131
1530993217,12cyclictest0-21swapper/300:00:153
151599324,17cyclictest27346irq/40-dwc2_hso19:15:142
145499324,20cyclictest21848-21ssh22:05:120
145499320,3cyclictest131rcu_preempt21:40:250
153099313,16cyclictest35966irq/58-eth023:55:133
1530993117,9cyclictest0-21swapper/323:45:113
151599313,20cyclictest27346irq/40-dwc2_hso22:55:162
145499314,19cyclictest32351-21sh23:05:140
153099303,14cyclictest35966irq/58-eth021:45:133
1530993018,10cyclictest0-21swapper/322:15:123
153099301,21cyclictest35966irq/58-eth023:20:143
151599305,15cyclictest23326-21sh23:00:102
151599303,16cyclictest28712-21ssh00:15:142
151599301,17cyclictest14658-21ssh21:25:132
153099294,15cyclictest35966irq/58-eth000:30:133
153099293,19cyclictest35966irq/58-eth023:05:143
1530992923,3cyclictest0-21swapper/300:35:133
153099292,21cyclictest0-21swapper/323:00:123
151599295,12cyclictest27-21ksoftirqd/220:35:152
151599294,12cyclictest27346irq/40-dwc2_hso21:40:122
148199292,19cyclictest5309-21ssh00:20:151
148199291,22cyclictest21868-21ssh22:05:141
145499295,15cyclictest5425-21ssh22:50:140
1454992921,4cyclictest0-21swapper/022:10:140
153099283,19cyclictest35966irq/58-eth022:30:113
1530992817,8cyclictest35966irq/58-eth021:25:133
1530992817,8cyclictest0-21swapper/300:15:153
1530992816,9cyclictest0-21swapper/322:05:123
151599288,11cyclictest0-21swapper/219:40:142
151599285,13cyclictest27346irq/40-dwc2_hso21:05:112
151599284,16cyclictest27346irq/40-dwc2_hso22:10:132
151599283,18cyclictest27346irq/40-dwc2_hso20:50:112
151599283,14cyclictest27346irq/40-dwc2_hso19:25:132
148199283,5cyclictest21-21ksoftirqd/120:50:111
1454992823,3cyclictest0-21swapper/023:45:120
1530992717,4cyclictest0-21swapper/322:00:163
1530992714,8cyclictest35966irq/58-eth023:10:143
1530992714,8cyclictest35966irq/58-eth023:10:143
151599273,18cyclictest27346irq/40-dwc2_hso21:35:152
151599272,16cyclictest27346irq/40-dwc2_hso19:20:132
151599271,22cyclictest16176-21cut23:50:162
1481992712,4cyclictest131rcu_preempt00:30:131
145499273,19cyclictest7486-21ssh22:15:130
145499272,18cyclictest32215-21ssh00:35:150
1454992714,9cyclictest28860-21ssh22:45:150
1454992714,5cyclictest12-21ksoftirqd/000:25:360
1454992713,3cyclictest0-21swapper/022:03:290
145499271,17cyclictest18247-21sh21:45:120
1530992616,7cyclictest35966irq/58-eth023:35:143
153099261,16cyclictest35966irq/58-eth000:05:153
151599262,15cyclictest27346irq/40-dwc2_hso19:10:142
148199266,4cyclictest131rcu_preempt23:10:141
148199266,4cyclictest131rcu_preempt23:10:131
148199261,3cyclictest131rcu_preempt21:10:131
145499269,9cyclictest12-21ksoftirqd/000:33:390
145499269,3cyclictest131rcu_preempt21:32:340
145499265,18cyclictest20023-21ssh21:10:130
145499263,3cyclictest131rcu_preempt22:30:140
153099255,16cyclictest35966irq/58-eth021:30:133
153099254,10cyclictest0-21swapper/320:05:133
153099252,18cyclictest35966irq/58-eth022:40:133
153099252,14cyclictest35966irq/58-eth022:20:153
153099252,14cyclictest35966irq/58-eth022:20:143
1530992518,5cyclictest35966irq/58-eth000:20:153
1530992517,5cyclictest35966irq/58-eth023:25:133
1530992513,8cyclictest0-21swapper/323:40:153
153099251,15cyclictest0-21swapper/319:20:123
151599254,16cyclictest27346irq/40-dwc2_hso23:45:142
151599252,18cyclictest3390-21sh23:25:142
1515992517,5cyclictest16953-21ssh23:30:312
1515992512,8cyclictest27-21ksoftirqd/200:00:162
148199258,12cyclictest131rcu_preempt00:19:031
148199255,14cyclictest131rcu_preempt23:21:471
148199252,8cyclictest0-21swapper/120:00:161
148199251,18cyclictest21391-21ssh23:35:151
145499258,4cyclictest12-21ksoftirqd/021:55:140
145499252,14cyclictest151rcuc/000:00:150
1454992516,6cyclictest12-21ksoftirqd/019:15:140
1454992514,5cyclictest12-21ksoftirqd/023:13:100
1454992514,5cyclictest12-21ksoftirqd/023:13:090
1454992510,4cyclictest0-21swapper/022:40:140
153099247,11cyclictest35966irq/58-eth021:50:113
1530992419,2cyclictest0-21swapper/319:15:133
1530992417,4cyclictest0-21swapper/323:15:143
151599243,17cyclictest27346irq/40-dwc2_hso00:05:162
148199248,10cyclictest0-21swapper/121:56:491
148199245,13cyclictest131rcu_preempt22:00:041
148199244,11cyclictest0-21swapper/120:20:121
148199242,14cyclictest14298-21ssh00:25:141
148199241,6cyclictest0-21swapper/119:45:131
1481992411,10cyclictest201rcuc/100:00:141
145499249,4cyclictest12-21ksoftirqd/022:20:400
145499249,4cyclictest12-21ksoftirqd/022:20:390
145499246,3cyclictest131rcu_preempt20:30:280
145499245,5cyclictest131rcu_preempt20:00:130
145499245,3cyclictest131rcu_preempt20:45:150
1454992415,6cyclictest4101-21ssh23:25:180
1454992413,7cyclictest12-21ksoftirqd/023:50:130
1454992411,7cyclictest0-21swapper/023:03:260
1454992410,8cyclictest12-21ksoftirqd/021:16:110
1454992410,8cyclictest12-21ksoftirqd/021:16:100
153099232,11cyclictest35966irq/58-eth019:35:143
1530992316,4cyclictest0-21swapper/319:30:133
153099231,16cyclictest35966irq/58-eth019:50:133
151599233,14cyclictest27346irq/40-dwc2_hso20:10:132
151599231,8cyclictest0-21swapper/223:40:152
1515992312,9cyclictest32517-21ssh21:51:042
1515992312,9cyclictest2368-21ssh22:46:532
151599231,15cyclictest25316-21find22:25:112
148199238,7cyclictest0-21swapper/119:15:161
148199237,9cyclictest21-21ksoftirqd/121:39:201
148199237,3cyclictest24944-21ssh22:40:461
148199235,9cyclictest21-21ksoftirqd/100:35:271
148199231,4cyclictest22701-21kworker/1:123:57:201
148199231,3cyclictest0-21swapper/121:18:521
148199231,3cyclictest0-21swapper/121:18:511
145499238,8cyclictest12-21ksoftirqd/019:40:170
145499238,8cyclictest12-21ksoftirqd/000:22:010
145499238,4cyclictest12-21ksoftirqd/023:41:240
145499238,4cyclictest12-21ksoftirqd/021:35:050
145499238,4cyclictest12-21ksoftirqd/020:40:160
145499237,7cyclictest12-21ksoftirqd/000:16:580
145499237,6cyclictest12-21ksoftirqd/023:19:020
145499233,7cyclictest12-21ksoftirqd/021:53:000
145499233,5cyclictest12-21ksoftirqd/023:57:380
145499231,8cyclictest14795-21fschecks_count21:25:150
1530992216,3cyclictest0-21swapper/320:50:123
1530992216,3cyclictest0-21swapper/319:45:143
1530992213,4cyclictest0-21swapper/319:25:133
1530992212,7cyclictest0-21swapper/321:15:223
1530992212,7cyclictest0-21swapper/321:15:213
151599224,10cyclictest26351-21usage_EthcTask22:05:302
151599223,12cyclictest27346irq/40-dwc2_hso23:55:142
151599222,6cyclictest0-21swapper/200:25:042
151599222,13cyclictest27346irq/40-dwc2_hso21:30:122
151599221,9cyclictest23290-21ssh00:30:152
151599221,15cyclictest19664-21ssh00:10:162
148199227,2cyclictest131rcu_preempt19:40:021
148199226,3cyclictest131rcu_preempt22:10:561
148199221,7cyclictest0-21swapper/121:30:131
148199221,3cyclictest0-21swapper/122:29:501
148199221,16cyclictest23344-21ssh23:00:111
148199221,15cyclictest10747-21kworker/1:222:52:371
148199221,13cyclictest0-21swapper/120:30:281
1481992210,4cyclictest21-21ksoftirqd/122:30:271
1481992210,2cyclictest21-21ksoftirqd/120:40:241
148199220,12cyclictest7173-21wc23:45:161
145499229,7cyclictest12-21ksoftirqd/020:21:450
145499228,8cyclictest12-21ksoftirqd/022:57:350
145499228,5cyclictest0-21swapper/000:10:000
145499227,3cyclictest131rcu_preempt21:00:210
145499226,3cyclictest131rcu_preempt00:10:320
145499225,9cyclictest12-21ksoftirqd/022:35:390
145499224,3cyclictest131rcu_preempt23:20:220
153099215,10cyclictest0-21swapper/322:55:153
1530992116,2cyclictest0-21swapper/319:40:123
1530992115,3cyclictest0-21swapper/322:50:143
1530992114,2cyclictest0-21swapper/300:25:123
1530992111,7cyclictest0-21swapper/322:46:283
151599212,13cyclictest29011-21missed_timers19:35:242
151599212,13cyclictest27346irq/40-dwc2_hso21:20:132
151599211,9cyclictest505266ssh22:30:222
151599211,8cyclictest13385-21kthreadcore19:55:202
151599211,11cyclictest3584-21awk23:05:262
1515992110,8cyclictest494-21kthreadcore00:35:202
1515992110,8cyclictest29132-21sh21:15:162
1515992110,8cyclictest29132-21sh21:15:152
148199218,8cyclictest21-21ksoftirqd/120:05:251
148199217,7cyclictest0-21swapper/119:55:271
148199216,7cyclictest21-21ksoftirqd/121:20:261
148199215,8cyclictest21-21ksoftirqd/120:15:191
148199212,3cyclictest131rcu_preempt21:00:231
148199212,10cyclictest21-21ksoftirqd/122:47:441
1481992111,7cyclictest31012-21sendmail_mailqu21:50:271
1481992110,8cyclictest10920-21apt-get22:35:141
1481992110,3cyclictest0-21swapper/120:55:251
145499214,5cyclictest12-21ksoftirqd/022:25:280
145499214,3cyclictest131rcu_preempt20:15:190
145499212,4cyclictest131rcu_preempt23:35:140
145499211,3cyclictest131rcu_preempt21:20:270
145499211,2cyclictest131rcu_preempt19:45:230
1454992112,3cyclictest0-21swapper/023:30:090
145499211,14cyclictest31706-21ls20:15:010
153099209,8cyclictest0-21swapper/322:10:203
1530992015,3cyclictest0-21swapper/300:10:153
1530992014,4cyclictest0-21swapper/320:35:143
1530992011,7cyclictest0-21swapper/322:26:393
1530992010,7cyclictest0-21swapper/323:54:563
1530992010,7cyclictest0-21swapper/323:30:273
151599208,4cyclictest7051-21kthreadcore20:55:182
151599203,9cyclictest13268-21kthreadcore21:00:232
151599203,4cyclictest0-21swapper/220:05:132
151599203,10cyclictest32731-21kthreadcore20:15:202
151599202,8cyclictest8788-21ssh22:15:202
151599202,8cyclictest16738-21kthreadcore20:30:212
151599201,9cyclictest3943-21fschecks_count21:55:152
151599201,9cyclictest20227-21grep21:10:172
151599201,9cyclictest19887-21kthreadcore20:00:222
151599201,9cyclictest10882-21kthreadcore19:50:222
151599201,11cyclictest4285-21apt-get20:20:122
151599201,10cyclictest7083-21kthreadcore00:20:232
148199208,8cyclictest201rcuc/120:10:131
148199208,3cyclictest21-21ksoftirqd/122:15:131
148199203,4cyclictest0-21swapper/119:55:021
148199201,11cyclictest18866-21sed21:05:291
145499209,8cyclictest17202-21apt-get20:00:010
145499209,3cyclictest131rcu_preempt20:35:290
145499204,4cyclictest12-21ksoftirqd/019:30:230
145499201,7cyclictest0-21swapper/020:55:120
1454992014,3cyclictest15870-21munin-plugin-st19:25:000
1454992011,6cyclictest7990-21cut19:50:160
1454992011,3cyclictest12-21ksoftirqd/019:10:170
145499201,11cyclictest5193-21cron20:55:020
153099199,7cyclictest0-21swapper/321:35:313
1530991914,3cyclictest0-21swapper/321:05:133
1530991914,2cyclictest0-21swapper/320:15:133
1530991911,4cyclictest0-21swapper/320:00:273
151599192,8cyclictest25102-21usage_EthcTask19:30:322
151599191,9cyclictest3198-21wc19:45:162
151599191,8cyclictest29077-21kthreadcore20:45:212
151599191,15cyclictest28084-21cat20:45:012
151599191,14cyclictest9094-21cpu20:25:182
151599191,10cyclictest18307-21/usr/sbin/munin23:15:202
148199199,2cyclictest21-21ksoftirqd/120:45:161
148199199,2cyclictest21-21ksoftirqd/119:20:291
148199193,7cyclictest21-21ksoftirqd/119:40:191
148199193,5cyclictest21-21ksoftirqd/119:10:031
145499194,3cyclictest131rcu_preempt19:25:120
145499193,9cyclictest131rcu_preempt20:25:150
145499193,5cyclictest12-21ksoftirqd/019:35:220
145499191,7cyclictest0-21swapper/021:07:480
153099189,6cyclictest0-21swapper/320:55:153
153099188,7cyclictest0-21swapper/320:40:163
153099188,7cyclictest0-21swapper/320:10:273
153099188,7cyclictest0-21swapper/319:55:163
153099182,10cyclictest0-21swapper/319:10:033
1530991814,2cyclictest0-21swapper/321:00:123
1530991814,2cyclictest0-21swapper/320:45:133
148199188,3cyclictest131rcu_preempt20:25:171
148199187,2cyclictest131rcu_preempt19:10:021
148199183,3cyclictest21798-21cut19:30:201
145499184,3cyclictest131rcu_preempt20:05:210
153099179,6cyclictest0-21swapper/321:40:033
1530991713,2cyclictest0-21swapper/320:30:133
1530991713,2cyclictest0-21swapper/320:20:113
145499176,3cyclictest131rcu_preempt19:10:020
1530991612,2cyclictest0-21swapper/320:25:163
151599151,7cyclictest1559-21latency_hist19:10:012
153099139,2cyclictest0-21swapper/319:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional