You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-20 - 18:46

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot4s.osadl.org (updated Sun Sep 20, 2026 12:45:11)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
18778996615,11cyclictest12-21ksoftirqd/012:05:120
1879399564,31cyclictest0-21swapper/211:15:102
18793995023,9cyclictest0-21swapper/210:45:132
1877899500,34cyclictest11134-21ssh10:25:120
18793994733,9cyclictest27-21ksoftirqd/211:55:132
18793994733,9cyclictest27-21ksoftirqd/211:55:132
1880099464,27cyclictest35966irq/58-eth010:35:153
18787994515,21cyclictest21-21ksoftirqd/110:55:131
1877899452,36cyclictest30745-21ssh10:00:120
1880099443,24cyclictest35966irq/58-eth011:30:123
18793994420,7cyclictest0-21swapper/209:35:152
1877899441,35cyclictest11441-21ssh11:55:120
1877899441,35cyclictest11441-21ssh11:55:110
18793994213,22cyclictest2456-21sh11:50:152
1880099414,28cyclictest35966irq/58-eth011:45:143
1880099414,20cyclictest35966irq/58-eth010:50:123
1878799400,37cyclictest9424-21ssh11:00:141
1880099394,24cyclictest35966irq/58-eth010:00:133
18800993915,15cyclictest35966irq/58-eth009:00:143
18800993915,15cyclictest35966irq/58-eth009:00:143
1880099383,28cyclictest35966irq/58-eth010:15:143
1877899383,24cyclictest27527-21ssh11:10:140
1880099361,22cyclictest35966irq/58-eth012:05:123
1879399368,22cyclictest27-21ksoftirqd/208:53:252
1879399368,21cyclictest27-21ksoftirqd/209:20:252
1879399362,26cyclictest18970-21df_inode12:35:152
1877899363,22cyclictest32335-21ssh09:25:140
1880099352,22cyclictest35966irq/58-eth012:10:173
1879399353,9cyclictest0-21swapper/209:49:422
1878799350,26cyclictest0-21swapper/109:15:141
1880099344,25cyclictest35966irq/58-eth010:05:123
1880099342,21cyclictest35966irq/58-eth009:40:143
1879399341,22cyclictest16400-21rm10:10:132
1880099332,19cyclictest35966irq/58-eth010:30:143
1879399334,14cyclictest24485-21kworker/u8:010:40:142
1879399332,9cyclictest6006-21ssh12:10:162
18793993313,11cyclictest18768-21cyclictest10:50:132
18787993320,8cyclictest32602-21apt-get09:05:131
1880099322,10cyclictest0-21swapper/309:05:043
1879399322,21cyclictest27346irq/40-dwc2_hso11:40:142
1879399321,25cyclictest25868-21ssh11:45:132
1878799324,6cyclictest21-21ksoftirqd/110:45:011
1879399312,20cyclictest27346irq/40-dwc2_hso08:45:132
1877899312,23cyclictest8926-21ssh09:30:130
1877899311,23cyclictest12691-21ssh09:50:120
18778993110,5cyclictest0-21swapper/011:00:130
1877899310,3cyclictest131rcu_preempt10:15:300
1880099304,14cyclictest35966irq/58-eth012:00:123
1879399304,18cyclictest0-21swapper/208:10:152
1879399304,18cyclictest0-21swapper/208:10:152
1879399303,20cyclictest27346irq/40-dwc2_hso08:05:142
1879399303,18cyclictest27346irq/40-dwc2_hso10:00:132
1879399302,21cyclictest0-21swapper/207:25:142
1879399302,21cyclictest0-21swapper/207:25:132
1878799303,4cyclictest21-21ksoftirqd/112:15:121
1877899301,8cyclictest12-21ksoftirqd/010:55:130
18778993012,7cyclictest12-21ksoftirqd/012:00:120
18800992915,9cyclictest35966irq/58-eth010:45:143
1879399299,12cyclictest1-21systemd07:19:112
1879399295,17cyclictest27346irq/40-dwc2_hso12:15:142
1879399294,14cyclictest27346irq/40-dwc2_hso12:30:152
1879399293,9cyclictest11139-21ssh10:25:142
1879399292,21cyclictest8778-21ntp_states09:10:242
1879399292,21cyclictest16665-21ssh11:03:252
1879399291,10cyclictest30032-21irqrtprio12:05:192
1878799294,5cyclictest21-21ksoftirqd/111:40:021
18787992913,4cyclictest21-21ksoftirqd/112:38:081
1877899295,5cyclictest12-21ksoftirqd/010:35:140
1877899293,4cyclictest12-21ksoftirqd/011:25:140
1877899292,20cyclictest21584-21ntp_192.168.11409:35:250
1877899292,19cyclictest4439-21ntp_states07:25:250
1877899292,19cyclictest4439-21ntp_states07:25:250
18800992813,11cyclictest0-21swapper/312:15:133
1879399284,14cyclictest27346irq/40-dwc2_hso10:20:132
1879399283,6cyclictest18506-21ssh11:05:142
1879399282,20cyclictest25467-21ssh10:15:142
1879399282,19cyclictest0-21swapper/208:25:152
1879399282,18cyclictest12591-21ntp_states09:30:262
1879399282,17cyclictest20185-21rm10:30:142
1877899287,13cyclictest131rcu_preempt09:55:120
1877899281,22cyclictest5133-21ssh09:10:130
1879399274,14cyclictest27346irq/40-dwc2_hso08:00:142
1879399272,20cyclictest0-21swapper/207:35:142
1879399272,19cyclictest19959-21ssh09:53:252
1879399272,18cyclictest0-21swapper/207:50:142
1879399271,16cyclictest27346irq/40-dwc2_hso08:35:132
1878799272,19cyclictest31388-21ssh09:24:251
18787992718,2cyclictest0-21swapper/110:35:141
1878799271,7cyclictest21-21ksoftirqd/110:25:161
18787992713,8cyclictest21-21ksoftirqd/111:20:111
18787992711,5cyclictest21-21ksoftirqd/109:31:061
1877899278,9cyclictest12-21ksoftirqd/010:47:360
1877899276,3cyclictest131rcu_preempt08:45:270
1877899275,3cyclictest131rcu_preempt11:45:130
1877899274,14cyclictest7877-21ssh11:35:120
1877899271,7cyclictest6238-21/usr/sbin/munin11:50:270
18778992713,4cyclictest12-21ksoftirqd/012:15:240
18778992710,9cyclictest0-21swapper/012:29:280
1879399263,16cyclictest27346irq/40-dwc2_hso12:00:112
1879399263,16cyclictest27346irq/40-dwc2_hso08:55:132
1879399261,9cyclictest20286-21kthreadcore07:10:202
1878799267,5cyclictest21-21ksoftirqd/110:46:381
1878799264,5cyclictest21-21ksoftirqd/109:30:011
1878799264,5cyclictest21-21ksoftirqd/108:00:181
1878799263,6cyclictest5259-21ssh11:50:231
1878799263,4cyclictest21-21ksoftirqd/112:34:021
18787992616,6cyclictest0-21swapper/107:45:141
18787992614,8cyclictest22908-21apt-get08:55:141
18787992610,11cyclictest7132-21apt-get08:05:141
1877899269,4cyclictest12-21ksoftirqd/012:38:020
1877899263,3cyclictest131rcu_preempt10:30:130
18778992612,6cyclictest0-21swapper/011:20:510
18778992611,5cyclictest0-21swapper/010:13:270
1880099254,12cyclictest0-21swapper/311:50:143
1880099253,10cyclictest0-21swapper/311:55:073
1880099253,10cyclictest0-21swapper/311:55:063
18800992512,9cyclictest35966irq/58-eth009:55:133
1880099251,17cyclictest35966irq/58-eth009:45:143
1879399259,12cyclictest1766-21kthreadcore09:25:232
1879399256,9cyclictest25639-21find09:55:272
1879399254,11cyclictest27346irq/40-dwc2_hso07:40:122
18793992512,8cyclictest5301-21latency_hist10:40:032
18793992511,5cyclictest32181-21kthreadcore11:30:202
1878799254,6cyclictest21-21ksoftirqd/110:05:271
1878799254,5cyclictest21-21ksoftirqd/110:50:241
1878799253,5cyclictest21-21ksoftirqd/111:14:141
1878799253,3cyclictest0-21swapper/109:50:221
1878799252,5cyclictest21-21ksoftirqd/112:09:151
1878799252,4cyclictest21-21ksoftirqd/109:10:271
18787992517,3cyclictest11967-21apt-get08:10:151
18787992517,3cyclictest11967-21apt-get08:10:141
1878799251,6cyclictest21-21ksoftirqd/107:30:271
1878799251,6cyclictest21-21ksoftirqd/107:30:271
18787992514,3cyclictest4733-21diskmemload11:30:311
18787992510,4cyclictest21-21ksoftirqd/110:17:201
1877899259,9cyclictest0-21swapper/009:41:430
1877899259,7cyclictest12-21ksoftirqd/011:17:130
1877899255,3cyclictest131rcu_preempt07:30:200
1877899255,3cyclictest131rcu_preempt07:30:200
1877899254,5cyclictest12-21ksoftirqd/010:05:200
1877899252,3cyclictest131rcu_preempt09:20:190
18778992518,3cyclictest0-21swapper/008:55:140
18778992514,3cyclictest26915-21ssh11:09:470
1877899251,19cyclictest12021-21ssh09:49:410
18778992511,5cyclictest12-21ksoftirqd/011:41:150
18778992510,7cyclictest0-21swapper/012:24:250
18778992510,4cyclictest12-21ksoftirqd/010:40:230
18778992510,4cyclictest12-21ksoftirqd/007:40:210
1880099240,18cyclictest35966irq/58-eth011:00:153
1879399249,6cyclictest12690-21cat11:35:312
1879399241,9cyclictest735-21ssh12:25:142
1879399241,12cyclictest7381-21rm10:05:122
1878799249,7cyclictest21-21ksoftirqd/112:25:081
1878799245,3cyclictest131rcu_preempt11:15:141
1878799244,6cyclictest21-21ksoftirqd/109:55:151
1878799244,5cyclictest21-21ksoftirqd/111:45:161
1878799244,5cyclictest21-21ksoftirqd/111:40:231
1878799244,5cyclictest21-21ksoftirqd/110:30:221
1878799243,6cyclictest21-21ksoftirqd/111:25:241
1878799243,6cyclictest21-21ksoftirqd/107:25:131
1878799243,6cyclictest21-21ksoftirqd/107:25:121
1878799243,5cyclictest21-21ksoftirqd/112:20:211
1878799243,2cyclictest131rcu_preempt09:40:121
1878799242,5cyclictest21-21ksoftirqd/110:20:311
1878799242,4cyclictest21-21ksoftirqd/109:45:311
1878799241,5cyclictest21-21ksoftirqd/111:55:141
1878799241,5cyclictest21-21ksoftirqd/111:55:141
1877899247,6cyclictest12-21ksoftirqd/009:10:020
1877899244,5cyclictest12-21ksoftirqd/007:50:160
1877899241,8cyclictest16786-21/usr/sbin/munin08:15:270
18778992413,4cyclictest0-21swapper/011:30:260
18778992410,4cyclictest12-21ksoftirqd/008:50:180
1880099233,13cyclictest0-21swapper/309:53:253
1880099233,13cyclictest0-21swapper/309:24:253
1879399232,6cyclictest0-21swapper/210:59:552
1879399232,6cyclictest0-21swapper/207:30:042
1879399232,6cyclictest0-21swapper/207:30:032
1879399232,16cyclictest27052-21apt-get09:40:132
1879399231,9cyclictest2870366ssh12:20:312
1879399231,15cyclictest30867-21grep11:30:012
1879399231,12cyclictest28035-21cut11:10:192
1878799236,3cyclictest131rcu_preempt08:20:011
1878799235,4cyclictest21-21ksoftirqd/112:10:231
1878799234,5cyclictest21-21ksoftirqd/110:10:181
1878799233,5cyclictest21-21ksoftirqd/110:00:301
1878799233,5cyclictest21-21ksoftirqd/109:35:221
1878799233,5cyclictest21-21ksoftirqd/107:50:161
1878799233,4cyclictest21-21ksoftirqd/112:02:561
1878799233,11cyclictest21-21ksoftirqd/107:45:021
1878799232,4cyclictest21-21ksoftirqd/107:10:011
18787992318,3cyclictest201rcuc/111:07:071
18787992316,3cyclictest27709-21apt-get09:00:151
18787992316,3cyclictest27709-21apt-get09:00:141
1877899239,4cyclictest131rcu_preempt10:50:130
1877899234,2cyclictest131rcu_preempt10:21:280
1877899233,4cyclictest12-21ksoftirqd/009:15:020
18778992311,4cyclictest12-21ksoftirqd/012:30:030
18778992311,4cyclictest12-21ksoftirqd/012:10:190
1880099228,9cyclictest0-21swapper/310:15:023
1879399223,12cyclictest27346irq/40-dwc2_hso09:05:142
1879399223,10cyclictest19827-21sshd09:16:262
1879399221,6cyclictest0-21swapper/207:55:142
18793992212,8cyclictest8833-21irqcore08:40:192
1879399221,17cyclictest2595-21grep08:30:302
1879399221,16cyclictest24144-21kthreadcore08:20:202
1879399221,11cyclictest22084-21kthreadcore07:45:192
1879399221,11cyclictest15823-21kthreadcore11:20:202
1878799224,5cyclictest21-21ksoftirqd/107:35:251
1878799224,4cyclictest21-21ksoftirqd/108:50:271
1878799223,4cyclictest21-21ksoftirqd/108:40:221
1878799222,4cyclictest21-21ksoftirqd/107:20:181
1878799221,4cyclictest21-21ksoftirqd/107:15:201
1877899229,4cyclictest12-21ksoftirqd/007:45:220
1877899229,2cyclictest131rcu_preempt08:20:170
1877899228,8cyclictest12-21ksoftirqd/007:15:270
1877899226,7cyclictest12-21ksoftirqd/007:20:270
1877899224,3cyclictest131rcu_preempt08:10:200
1877899224,3cyclictest131rcu_preempt08:10:190
1877899222,14cyclictest2194-21ntp_192.168.11408:30:250
1877899222,10cyclictest12-21ksoftirqd/009:00:220
1877899222,10cyclictest12-21ksoftirqd/009:00:220
1877899221,3cyclictest131rcu_preempt07:15:020
1877899220,10cyclictest12-21ksoftirqd/007:35:200
1880099212,14cyclictest0-21swapper/311:40:143
1880099212,13cyclictest0-21swapper/310:20:133
1879399216,7cyclictest31519-21grep09:00:292
1879399216,7cyclictest31519-21grep09:00:282
1879399211,16cyclictest18901-21latency_hist07:10:022
1879399211,10cyclictest32089-21wc07:20:262
1878799213,4cyclictest21-21ksoftirqd/108:20:201
1878799213,4cyclictest21-21ksoftirqd/107:55:221
1878799213,4cyclictest21-21ksoftirqd/107:10:191
1878799212,5cyclictest21-21ksoftirqd/108:30:171
1878799211,4cyclictest21-21ksoftirqd/108:35:221
1877899219,4cyclictest12-21ksoftirqd/008:40:220
1877899216,9cyclictest131rcu_preempt08:25:280
1877899213,3cyclictest131rcu_preempt08:35:290
1880099202,12cyclictest0-21swapper/311:05:143
1880099202,10cyclictest0-21swapper/307:25:143
1880099202,10cyclictest0-21swapper/307:25:143
18800992010,8cyclictest0-21swapper/311:35:413
18800992010,8cyclictest0-21swapper/310:40:043
1879399201,16cyclictest16957-21fschecks_time08:15:162
1878799203,4cyclictest26418-21apt-get08:25:131
1878799202,5cyclictest21-21ksoftirqd/108:45:271
1877899208,4cyclictest12-21ksoftirqd/008:00:170
1877899205,8cyclictest131rcu_preempt08:05:290
1877899204,8cyclictest131rcu_preempt07:55:200
1880099199,6cyclictest35966irq/58-eth011:20:113
1880099197,8cyclictest35966irq/58-eth011:25:153
1880099193,11cyclictest0-21swapper/312:25:123
1880099193,11cyclictest0-21swapper/309:35:263
1880099193,10cyclictest0-21swapper/308:25:243
1880099192,11cyclictest0-21swapper/308:35:123
1880099192,11cyclictest0-21swapper/308:10:253
1880099192,11cyclictest0-21swapper/308:10:243
1880099192,11cyclictest0-21swapper/308:00:243
1880099191,12cyclictest0-21swapper/312:35:153
1880099191,12cyclictest0-21swapper/308:50:013
18800991910,7cyclictest0-21swapper/311:15:293
1880099189,6cyclictest0-21swapper/312:21:593
1880099189,6cyclictest0-21swapper/310:57:523
1880099189,6cyclictest0-21swapper/309:30:183
1880099183,7cyclictest0-21swapper/310:25:133
1880099183,10cyclictest0-21swapper/309:29:253
1880099183,10cyclictest0-21swapper/309:10:253
1880099183,10cyclictest0-21swapper/308:58:263
1880099183,10cyclictest0-21swapper/308:40:253
1880099183,10cyclictest0-21swapper/307:50:253
1880099183,10cyclictest0-21swapper/307:10:253
1880099182,11cyclictest0-21swapper/308:53:253
1880099182,10cyclictest0-21swapper/307:55:153
1880099182,10cyclictest0-21swapper/307:40:123
18800991810,5cyclictest35966irq/58-eth012:30:143
1880099178,6cyclictest35966irq/58-eth011:10:153
1880099173,10cyclictest0-21swapper/308:15:253
1880099172,10cyclictest0-21swapper/308:30:163
1880099171,13cyclictest0-21swapper/309:15:273
1877899170,14cyclictest18852-21ls07:10:000
1880099162,9cyclictest0-21swapper/307:30:043
1880099162,9cyclictest0-21swapper/307:30:033
1880099151,9cyclictest0-21swapper/307:45:143
1880099150,10cyclictest0-21swapper/307:20:143
1880099145,3cyclictest0-21swapper/307:35:143
1880099141,7cyclictest0-21swapper/308:20:143
1880099134,5cyclictest0-21swapper/308:05:143
1880099131,7cyclictest0-21swapper/307:15:223
1880099112,7cyclictest0-21swapper/307:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional