You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-15 - 03:16

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot4s.osadl.org (updated Sat Aug 15, 2026 00:45:13)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
254239910238,19cyclictest131rcu_preempt20:45:122
254239910238,19cyclictest131rcu_preempt20:45:122
25423998114,12cyclictest131rcu_preempt20:20:122
25423996915,16cyclictest131rcu_preempt21:35:142
25423996312,42cyclictest131rcu_preempt21:30:092
2542399595,5cyclictest131rcu_preempt19:35:122
2540799577,36cyclictest12490-21ssh22:40:120
2542399532,7cyclictest131rcu_preempt21:40:132
2540799532,40cyclictest29546-21ssh21:20:120
2540799532,40cyclictest29546-21ssh21:20:110
2540799493,37cyclictest15202-21ssh21:30:100
2542399461,26cyclictest21504-21ssh22:45:102
25435994319,12cyclictest35966irq/58-eth021:25:113
2541499402,34cyclictest21514-21ssh22:45:111
2540799390,30cyclictest30534-21ssh22:50:140
2542399382,20cyclictest3745-21sshd00:05:132
2540799384,27cyclictest905-21ssh21:40:120
2543599374,22cyclictest35966irq/58-eth023:55:113
2543599371,19cyclictest35966irq/58-eth021:10:143
2541499369,17cyclictest13683-21ssh22:05:121
2540799364,21cyclictest17757-21ssh22:25:140
2540799363,27cyclictest16200-21ssh23:00:150
2543599352,14cyclictest35966irq/58-eth000:05:123
2541499351,28cyclictest22779-21sh22:10:141
2541499351,24cyclictest27227-21rm00:00:131
2542399342,6cyclictest131rcu_preempt21:50:132
2540799342,8cyclictest0-21swapper/021:00:130
2540799341,18cyclictest0-21swapper/022:45:100
25435993324,6cyclictest35966irq/58-eth022:50:133
2543599332,18cyclictest35966irq/58-eth021:40:123
2543599332,17cyclictest35966irq/58-eth022:40:123
2542399335,18cyclictest27346irq/40-dwc2_hso00:20:132
2542399331,15cyclictest18113-21ssh23:55:112
2541499332,22cyclictest4675-21ssh22:00:141
2541499331,19cyclictest6161-21ssh21:25:121
2540799337,9cyclictest12-21ksoftirqd/020:00:140
2542399323,24cyclictest131rcu_preempt00:38:502
2541499321,22cyclictest25163-21ssh23:05:141
2541499321,22cyclictest25163-21ssh23:05:131
2542399313,8cyclictest25395-21cyclictest23:00:152
2542399312,23cyclictest0-21swapper/219:40:142
2542399312,23cyclictest0-21swapper/219:40:132
2543599302,20cyclictest0-21swapper/319:35:103
2542399305,16cyclictest27346irq/40-dwc2_hso22:05:132
2542399302,21cyclictest0-21swapper/220:55:132
25423993014,11cyclictest1465-21kworker/2:121:20:122
25423993014,11cyclictest1465-21kworker/2:121:20:112
2541499302,19cyclictest28088-21ssh21:55:151
2541499301,22cyclictest26834-21ssh22:30:141
2540799308,17cyclictest18017-1kworker/u9:000:05:130
2540799302,25cyclictest26823-21ssh22:30:120
2542399292,21cyclictest20108-21ssh00:13:332
2541499295,7cyclictest21-21ksoftirqd/119:30:121
2541499295,7cyclictest21-21ksoftirqd/119:30:121
2541499292,20cyclictest10746-21ssh23:15:121
2541499291,7cyclictest21-21ksoftirqd/123:35:271
25414992914,10cyclictest21-21ksoftirqd/100:15:131
2541499290,22cyclictest0-21swapper/120:45:121
2541499290,22cyclictest0-21swapper/120:45:111
2540799292,8cyclictest12-21ksoftirqd/023:30:120
25435992814,8cyclictest35966irq/58-eth000:20:133
2542399283,15cyclictest27346irq/40-dwc2_hso19:50:132
2542399282,17cyclictest27346irq/40-dwc2_hso21:00:142
2541499286,5cyclictest21-21ksoftirqd/122:50:311
2541499283,9cyclictest17992-21tune2fs22:25:161
2541499282,19cyclictest7069-21ssh22:55:111
25414992819,3cyclictest30336-21apt-get19:15:141
25414992811,12cyclictest630-21Async09:36:311
2540799285,15cyclictest25395-21cyclictest19:50:120
2540799283,21cyclictest32584-21ssh23:45:130
2540799282,21cyclictest30795-21ssh00:20:140
25407992812,3cyclictest0-21swapper/021:37:330
2543599271,15cyclictest35966irq/58-eth022:20:123
2542399274,17cyclictest27346irq/40-dwc2_hso22:25:142
2541499276,5cyclictest21-21ksoftirqd/123:20:291
2541499274,5cyclictest21-21ksoftirqd/121:53:271
2541499273,5cyclictest21-21ksoftirqd/121:10:151
2541499273,15cyclictest9140-21ssh23:50:131
25414992719,4cyclictest897-21apt-get21:40:131
2540799279,9cyclictest12-21ksoftirqd/000:10:120
2540799274,4cyclictest131rcu_preempt00:03:210
2540799272,19cyclictest31873-21ssh22:15:130
2540799271,18cyclictest23418-21ssh23:20:250
2542399263,15cyclictest27346irq/40-dwc2_hso22:00:132
25423992618,5cyclictest32758-21ssh22:15:192
2542399261,21cyclictest11704-21cut21:10:162
2541499265,4cyclictest201rcuc/100:25:071
2541499264,6cyclictest21-21ksoftirqd/123:27:311
2541499263,6cyclictest21-21ksoftirqd/122:20:281
2541499262,18cyclictest25121-21sh00:35:141
25414992619,3cyclictest31848-21ssh22:15:121
2540799266,12cyclictest0-21swapper/019:35:130
2540799265,18cyclictest12-21ksoftirqd/021:55:150
2543599252,16cyclictest35966irq/58-eth023:00:153
2542399254,14cyclictest25557-21apt-get19:10:102
2542399253,13cyclictest27346irq/40-dwc2_hso20:50:132
2542399252,9cyclictest0-21swapper/220:10:142
2542399251,15cyclictest32553-21find23:45:122
2542399251,11cyclictest0-21swapper/223:29:472
2541499256,5cyclictest21-21ksoftirqd/121:30:191
2541499256,4cyclictest21-21ksoftirqd/122:40:151
2541499254,5cyclictest21-21ksoftirqd/123:32:031
2541499254,5cyclictest21-21ksoftirqd/123:11:261
2541499254,4cyclictest21-21ksoftirqd/123:40:171
2541499252,17cyclictest10836-21ssh00:08:131
25414992512,9cyclictest24299-21apt-get21:35:141
2540799258,12cyclictest151rcuc/022:35:140
2540799253,19cyclictest9953-21ssh21:45:110
2540799251,19cyclictest16191-21ssh00:30:120
25407992511,3cyclictest0-21swapper/023:41:410
2543599243,14cyclictest0-21swapper/300:10:133
2542399245,13cyclictest0-21swapper/219:45:092
2542399242,18cyclictest20291-21ssh23:36:412
2542399242,17cyclictest0-21swapper/200:01:112
25423992411,9cyclictest2510-21sendmail19:20:012
2541499249,10cyclictest24502-21apt-get20:50:131
2541499246,8cyclictest25395-21cyclictest00:13:541
2541499246,5cyclictest21-21ksoftirqd/123:45:231
2541499246,2cyclictest131rcu_preempt21:45:111
2541499245,14cyclictest21-21ksoftirqd/119:10:111
2541499244,6cyclictest21-21ksoftirqd/100:20:181
2541499244,4cyclictest21-21ksoftirqd/123:00:191
2541499243,4cyclictest21-21ksoftirqd/121:05:161
2541499243,4cyclictest21-21ksoftirqd/120:30:151
2541499242,4cyclictest21-21ksoftirqd/120:40:001
2540799243,3cyclictest131rcu_preempt00:15:520
2540799242,4cyclictest131rcu_preempt20:50:290
25407992414,8cyclictest12-21ksoftirqd/020:15:130
2543599232,11cyclictest35966irq/58-eth000:35:123
2542399238,11cyclictest12893-21kthreadcore20:35:242
2542399236,3cyclictest131rcu_preempt20:05:152
2542399236,14cyclictest9156-21ssh23:50:142
2542399234,15cyclictest27346irq/40-dwc2_hso00:25:142
2542399233,15cyclictest25162-21diskmemload23:05:142
2542399233,15cyclictest25162-21diskmemload23:05:142
2542399232,6cyclictest0-21swapper/223:32:412
2542399232,16cyclictest0-21swapper/200:18:142
2542399232,11cyclictest27863-21latency_hist20:20:002
2542399231,6cyclictest1725-21/usr/sbin/munin23:10:232
25423992312,4cyclictest0-21swapper/220:25:302
2542399231,18cyclictest22788-21ssh22:10:142
2541499237,4cyclictest21-21ksoftirqd/119:50:191
2541499236,4cyclictest0-21swapper/119:40:141
2541499236,4cyclictest0-21swapper/119:40:131
2541499236,3cyclictest131rcu_preempt00:34:231
2541499234,5cyclictest21-21ksoftirqd/121:00:301
2541499234,5cyclictest21-21ksoftirqd/119:45:091
2541499234,4cyclictest21-21ksoftirqd/121:00:021
2541499233,4cyclictest21-21ksoftirqd/123:55:151
2541499233,3cyclictest10963-21ntp_192.168.11419:25:251
2541499233,3cyclictest10963-21ntp_192.168.11419:25:251
2541499231,8cyclictest7202-21/usr/sbin/munin22:35:281
25414992316,3cyclictest17221-21apt-get19:35:121
2540799238,4cyclictest12-21ksoftirqd/022:10:140
25407992310,3cyclictest0-21swapper/023:52:510
2543599224,10cyclictest35966irq/58-eth022:05:133
2542399229,4cyclictest131rcu_preempt23:40:202
2542399228,10cyclictest9366-21kthreadcore22:55:192
2542399224,13cyclictest27-21ksoftirqd/223:20:122
2542399223,4cyclictest27-21ksoftirqd/221:45:202
2542399222,6cyclictest0-21swapper/222:40:122
2542399222,10cyclictest20532-21usage_Anybus00:30:302
2542399221,13cyclictest21660-21kthreadcore21:15:202
2541499224,3cyclictest131rcu_preempt19:20:251
2541499223,5cyclictest21-21ksoftirqd/120:15:151
2541499223,4cyclictest21-21ksoftirqd/120:40:141
2541499222,6cyclictest21-21ksoftirqd/120:20:251
2541499221,5cyclictest21-21ksoftirqd/121:24:441
2541499221,5cyclictest21-21ksoftirqd/121:24:441
2540799229,4cyclictest131rcu_preempt19:25:230
2540799229,4cyclictest131rcu_preempt19:25:230
2540799227,7cyclictest25395-21cyclictest21:05:130
2540799224,4cyclictest131rcu_preempt21:25:170
2540799222,2cyclictest131rcu_preempt22:04:500
2540799221,7cyclictest28108-21apt-get20:20:130
2540799221,17cyclictest3992-21apt-get19:55:110
2540799221,15cyclictest15088-21cut20:40:160
2540799221,10cyclictest18097-21ssh23:55:100
2543599216,12cyclictest33-21ksoftirqd/300:30:203
2543599215,9cyclictest33-21ksoftirqd/322:55:113
2543599214,9cyclictest0-21swapper/322:45:093
25435992113,6cyclictest35966irq/58-eth023:45:143
2543599211,14cyclictest0-21swapper/322:25:143
25435992110,8cyclictest0-21swapper/322:34:523
2542399212,15cyclictest930-21snmpd22:30:142
2542399212,11cyclictest16018-21cat19:30:262
2542399212,11cyclictest16018-21cat19:30:262
2542399211,17cyclictest27346irq/40-dwc2_hso23:15:112
2542399211,17cyclictest18168-21ls20:40:222
2542399211,16cyclictest4149-21uniq19:55:142
2541499213,4cyclictest21-21ksoftirqd/120:00:231
2540799213,6cyclictest30597-21ssh23:25:190
2540799213,15cyclictest151rcuc/023:10:100
2540799212,14cyclictest12-21ksoftirqd/000:38:490
2540799211,4cyclictest131rcu_preempt22:05:250
2540799211,14cyclictest30107-21latency_hist19:15:020
2543599209,7cyclictest35966irq/58-eth021:30:113
2543599208,9cyclictest0-21swapper/323:24:343
2543599204,11cyclictest35966irq/58-eth023:50:133
2543599204,11cyclictest33-21ksoftirqd/319:15:133
2543599202,11cyclictest35966irq/58-eth023:25:133
25435992013,4cyclictest0-21swapper/322:00:133
25435992012,5cyclictest0-21swapper/322:35:143
2542399202,5cyclictest0-21swapper/222:20:122
2542399202,10cyclictest0-21swapper/221:05:142
2542399201,9cyclictest7672-21fschecks_count19:25:162
2542399201,9cyclictest7672-21fschecks_count19:25:162
2542399201,9cyclictest3963-21kthreadcore19:20:192
2542399201,9cyclictest32544-21tr22:50:212
2542399201,16cyclictest29835-21tr21:55:212
2542399201,14cyclictest6140-21apt-get21:25:112
2542399201,10cyclictest9177-21head20:30:272
2542399201,10cyclictest4733-21cut22:35:212
2542399201,10cyclictest10194-21kthreadcore20:00:212
2541499207,4cyclictest21-21ksoftirqd/120:00:021
2541499203,5cyclictest21-21ksoftirqd/120:05:281
2541499202,4cyclictest21-21ksoftirqd/120:25:131
25414992014,4cyclictest0-21swapper/119:10:011
2541499201,3cyclictest21-21ksoftirqd/120:10:161
2540799203,7cyclictest7826-21kthreadcore22:55:170
2540799202,8cyclictest131rcu_preempt21:17:340
2540799202,2cyclictest131rcu_preempt23:05:160
2540799202,2cyclictest131rcu_preempt23:05:150
2540799201,5cyclictest53646usb-storage22:20:050
2540799201,10cyclictest6621-21unixbench_singl19:20:280
2540799200,18cyclictest29453-21uniq20:55:160
2543599199,7cyclictest35966irq/58-eth023:15:113
2543599193,11cyclictest0-21swapper/322:15:133
25435991913,4cyclictest0-21swapper/321:50:173
25435991911,5cyclictest35966irq/58-eth023:10:113
2543599191,12cyclictest0-21swapper/300:20:013
25435991910,4cyclictest0-21swapper/323:07:003
25435991910,4cyclictest0-21swapper/323:07:003
2540799199,8cyclictest0-21swapper/021:13:010
2540799195,3cyclictest131rcu_preempt20:05:160
2540799192,3cyclictest131rcu_preempt19:40:250
2540799192,3cyclictest131rcu_preempt19:40:250
2540799192,2cyclictest131rcu_preempt23:36:130
2540799191,2cyclictest131rcu_preempt19:45:180
25407991912,4cyclictest11031-21proc_pri00:25:270
25407991911,4cyclictest12-21ksoftirqd/023:15:240
2540799191,13cyclictest19034-21ssh21:50:130
2543599189,6cyclictest0-21swapper/300:28:593
2543599189,4cyclictest0-21swapper/321:46:253
2543599183,10cyclictest0-21swapper/323:36:423
2543599183,10cyclictest0-21swapper/323:31:403
2543599183,10cyclictest0-21swapper/300:01:113
2543599182,12cyclictest0-21swapper/322:10:133
2543599182,10cyclictest0-21swapper/319:40:143
2543599182,10cyclictest0-21swapper/319:40:143
25435991812,4cyclictest0-21swapper/321:17:043
2543599181,11cyclictest0-21swapper/320:30:133
2543599181,10cyclictest0-21swapper/321:35:143
25435991810,6cyclictest0-21swapper/321:23:413
25435991810,6cyclictest0-21swapper/321:23:403
25435991810,5cyclictest35966irq/58-eth023:40:133
2542399181,9cyclictest25429-21ls19:10:012
2540799185,6cyclictest0-21swapper/020:25:120
2540799181,2cyclictest131rcu_preempt19:30:190
2540799181,2cyclictest131rcu_preempt19:30:180
25407991811,5cyclictest1401-21open_inodes19:15:270
2540799180,2cyclictest131rcu_preempt20:45:220
2540799180,2cyclictest131rcu_preempt20:45:210
2543599178,6cyclictest35966irq/58-eth021:55:153
2543599172,10cyclictest0-21swapper/320:10:143
2543599171,13cyclictest0-21swapper/320:00:163
2540799174,10cyclictest19041-21kernelversion20:10:190
2540799172,10cyclictest0-21swapper/020:35:160
2540799170,2cyclictest131rcu_preempt20:30:090
2543599165,7cyclictest35966irq/58-eth020:55:123
2543599162,9cyclictest0-21swapper/319:25:143
2543599162,9cyclictest0-21swapper/319:25:133
2543599162,10cyclictest0-21swapper/320:25:133
2543599161,12cyclictest321rcuc/320:45:123
2543599161,12cyclictest321rcuc/320:45:123
2543599161,12cyclictest0-21swapper/319:45:133
25435991610,4cyclictest33-21ksoftirqd/320:35:323
2543599158,4cyclictest33-21ksoftirqd/320:50:293
2543599151,9cyclictest0-21swapper/320:20:153
2543599148,4cyclictest33-21ksoftirqd/319:10:253
2543599146,6cyclictest33-21ksoftirqd/320:15:233
2543599141,10cyclictest0-21swapper/321:05:293
2540799141,9cyclictest25430-21sort19:10:000
2543599132,9cyclictest0-21swapper/319:50:133
2543599131,7cyclictest0-21swapper/321:00:163
2543599128,2cyclictest33-21ksoftirqd/319:34:293
2543599128,2cyclictest33-21ksoftirqd/319:34:283
2543599121,9cyclictest0-21swapper/319:55:203
2543599121,8cyclictest0-21swapper/320:40:143
2543599112,7cyclictest0-21swapper/319:25:013
2543599111,7cyclictest0-21swapper/320:10:013
254359991,6cyclictest0-21swapper/319:10:003
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional