You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-27 - 21:17

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Mon Jul 27, 2026 12:45:11)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
21874998366,11cyclictest7820-21diskmemload09:15:132
21866997316,8cyclictest131rcu_preempt10:00:151
2187499533,43cyclictest18568-21ssh11:40:152
2187499532,33cyclictest2829-21ssh09:25:132
2187499532,33cyclictest2829-21ssh09:25:132
2185499524,41cyclictest18878-21ssh10:10:140
2186699494,6cyclictest131rcu_preempt07:45:141
2187499472,30cyclictest30865-21ssh12:05:132
2188299452,25cyclictest35966irq/58-eth012:15:143
2186699454,8cyclictest131rcu_preempt08:30:141
21866994525,11cyclictest131rcu_preempt10:05:141
2188299443,25cyclictest35966irq/58-eth011:05:123
2186699441,34cyclictest29879-21ssh09:40:131
21882994328,10cyclictest0-21swapper/310:15:133
2185499422,32cyclictest17181-21ssh09:15:130
2185499422,25cyclictest24144-21ssh11:25:160
2185499412,30cyclictest15127-21ssh11:20:130
2186699403,27cyclictest21-21ksoftirqd/107:35:141
2186699402,28cyclictest2195-21ssh12:25:151
21866994015,18cyclictest21-21ksoftirqd/110:40:141
2185499401,34cyclictest7465-21ssh12:10:130
2188299392,20cyclictest35966irq/58-eth007:40:133
2188299391,24cyclictest35966irq/58-eth010:00:133
2185499384,30cyclictest29632-21ssh11:10:130
21874993720,12cyclictest2194-21sh12:25:152
2186699371,29cyclictest15355-21ssh09:50:111
21866993622,9cyclictest0-21swapper/112:30:121
2186699362,22cyclictest20614-21ssh11:05:131
2185499363,24cyclictest844-21munin-node08:45:160
2185499363,24cyclictest24343-21ssh09:55:130
2187499352,24cyclictest0-21swapper/210:25:042
21854993526,4cyclictest0-21swapper/010:35:150
2188299344,14cyclictest35966irq/58-eth009:45:153
2187499344,13cyclictest26175-21sh09:20:122
2186699345,8cyclictest131rcu_preempt07:15:141
2185499346,22cyclictest13004-21ssh11:55:170
21854993420,6cyclictest0-21swapper/007:10:130
21854993415,12cyclictest12-21ksoftirqd/011:35:140
2186699332,23cyclictest22648-21ssh10:30:141
2188299324,17cyclictest35966irq/58-eth012:10:143
2188299324,16cyclictest35966irq/58-eth010:35:143
2187499322,20cyclictest930-21snmpd09:40:142
21874993211,14cyclictest271ksoftirqd/212:35:142
2186699323,16cyclictest27456-21ssh11:45:121
21866993212,4cyclictest0-21swapper/107:30:161
2186699320,28cyclictest15142-21ssh11:20:151
2187499312,22cyclictest0-21swapper/209:05:162
2186699313,23cyclictest25610-21ssh12:20:131
2188299302,15cyclictest0-21swapper/309:25:133
2188299302,15cyclictest0-21swapper/309:25:133
21882993016,10cyclictest35966irq/58-eth011:30:153
2187499304,19cyclictest27346irq/40-dwc2_hso11:15:152
2187499301,15cyclictest31573-21ssh10:35:142
2188299295,14cyclictest35966irq/58-eth010:55:133
2187499292,10cyclictest14683-21ssh11:35:512
21874992910,10cyclictest20225-21cron11:05:012
2186699292,21cyclictest26194-21ssh09:20:141
2188299282,15cyclictest35966irq/58-eth011:55:183
2188299282,13cyclictest35966irq/58-eth011:50:143
2187499283,13cyclictest27346irq/40-dwc2_hso07:55:122
2187499283,13cyclictest27346irq/40-dwc2_hso07:55:122
2187499282,10cyclictest0-21swapper/210:50:402
2187499281,11cyclictest2610-21ssh10:55:132
2185499283,5cyclictest12-21ksoftirqd/010:05:320
2185499282,20cyclictest28188-21ssh11:08:460
2185499281,19cyclictest11249-21ssh12:30:140
2187499279,10cyclictest23612-21tail07:45:202
2187499273,17cyclictest27346irq/40-dwc2_hso09:00:152
2187499272,15cyclictest8092-21apt-get10:40:132
21874992716,7cyclictest27-21ksoftirqd/208:00:122
2186699275,19cyclictest21-21ksoftirqd/111:35:131
2186699272,20cyclictest31976-21ssh10:51:461
21854992713,4cyclictest12-21ksoftirqd/009:10:190
2185499271,21cyclictest17072-21ssh10:45:140
2185499271,18cyclictest20861-21ssh09:35:140
2188299262,17cyclictest0-21swapper/311:40:143
2187499263,17cyclictest27346irq/40-dwc2_hso08:15:112
2187499263,15cyclictest27346irq/40-dwc2_hso12:00:122
2187499263,15cyclictest27346irq/40-dwc2_hso12:00:122
2187499262,9cyclictest3396-21ssh11:49:462
2187499262,19cyclictest0-21swapper/209:55:142
21874992618,5cyclictest11315-21ssh12:10:302
2186699264,15cyclictest18577-21ssh11:40:151
2186699263,18cyclictest7820-21diskmemload10:55:151
2186699261,5cyclictest17164-21apt-get09:15:131
21866992611,6cyclictest0-21swapper/109:30:291
2185499269,7cyclictest12-21ksoftirqd/011:46:570
2185499267,7cyclictest12-21ksoftirqd/011:42:120
2185499266,4cyclictest131rcu_preempt11:00:290
2185499266,3cyclictest131rcu_preempt12:20:190
2185499266,3cyclictest131rcu_preempt12:05:150
2185499263,5cyclictest12-21ksoftirqd/011:50:320
2185499262,18cyclictest4536-21ssh10:20:120
21854992613,5cyclictest12-21ksoftirqd/007:30:010
2188299252,17cyclictest0-21swapper/312:28:473
2187499253,14cyclictest27346irq/40-dwc2_hso11:30:152
2187499251,21cyclictest13379-21ls10:05:232
2187499251,14cyclictest8253-21ssh09:10:152
2187499251,10cyclictest11810-21irqrtprio12:30:182
2187499250,17cyclictest18985-21ls10:10:162
2186699251,16cyclictest20230-21ssh12:35:141
2185499255,4cyclictest12-21ksoftirqd/009:25:020
2185499254,8cyclictest12-21ksoftirqd/009:30:340
2185499254,5cyclictest12-21ksoftirqd/007:55:180
2185499254,5cyclictest12-21ksoftirqd/007:55:170
2185499254,12cyclictest12-21ksoftirqd/009:05:300
2185499253,17cyclictest584-21rm11:30:150
2185499252,18cyclictest6175-21ssh11:15:150
2185499251,4cyclictest12-21ksoftirqd/012:25:300
2185499250,9cyclictest18975-21ntp_192.168.11409:50:230
2188299243,13cyclictest0-21swapper/310:51:463
2188299242,15cyclictest0-21swapper/312:00:133
2188299242,15cyclictest0-21swapper/312:00:133
21882992413,3cyclictest0-21swapper/311:15:143
2188299241,15cyclictest0-21swapper/310:25:043
2188299241,14cyclictest35966irq/58-eth009:40:133
2187499243,16cyclictest27346irq/40-dwc2_hso09:35:122
2187499242,6cyclictest0-21swapper/210:46:462
2187499241,7cyclictest16584-21kthreadcore11:20:222
2187499241,14cyclictest20452-21cron09:35:002
2187499241,13cyclictest27346irq/40-dwc2_hso08:55:152
21866992412,10cyclictest0-21swapper/109:45:251
21866992411,3cyclictest0-21swapper/107:50:211
21866992411,11cyclictest0-21swapper/108:00:271
21866992410,9cyclictest21-21ksoftirqd/108:25:241
2185499246,3cyclictest131rcu_preempt10:54:570
2185499243,4cyclictest131rcu_preempt09:45:280
2185499243,3cyclictest131rcu_preempt10:56:130
2185499241,3cyclictest131rcu_preempt08:35:160
2188299232,16cyclictest35966irq/58-eth011:35:453
2187499237,13cyclictest16219-21kthreadcore11:55:252
2187499233,6cyclictest27-21ksoftirqd/210:15:132
2187499232,6cyclictest0-21swapper/211:14:472
2187499232,15cyclictest0-21swapper/211:50:162
2186699237,3cyclictest131rcu_preempt08:10:191
2186699233,11cyclictest21-21ksoftirqd/110:17:201
2186699231,16cyclictest22029-21apt-get07:10:121
2185499238,5cyclictest12-21ksoftirqd/012:03:470
2185499238,5cyclictest12-21ksoftirqd/012:03:460
2185499235,5cyclictest12-21ksoftirqd/009:25:200
2185499235,5cyclictest12-21ksoftirqd/009:25:190
2185499235,3cyclictest131rcu_preempt10:40:160
2185499235,3cyclictest131rcu_preempt10:16:460
2185499235,3cyclictest131rcu_preempt07:50:250
2185499234,4cyclictest131rcu_preempt08:20:010
2185499232,7cyclictest12-21ksoftirqd/012:15:230
2185499232,14cyclictest13463-21sh10:25:040
2185499231,4cyclictest12-21ksoftirqd/010:30:170
2185499231,19cyclictest29260-21ls08:55:240
2185499230,19cyclictest916-21find10:00:130
21882992213,6cyclictest0-21swapper/309:20:183
2187499228,11cyclictest29821-21irqcore08:25:182
2187499226,14cyclictest27346irq/40-dwc2_hso07:50:112
2187499224,15cyclictest23880-21kthreadcore10:30:202
2187499222,17cyclictest16229-21latency_hist08:45:012
2187499221,4cyclictest0-21swapper/209:55:012
2187499221,18cyclictest24236-21ntp_192.168.11411:05:252
2187499221,17cyclictest31648-21find07:20:122
2187499221,17cyclictest28992-21kthreadcore07:15:222
2187499221,17cyclictest20328-21ps12:15:282
21874992211,3cyclictest0-21swapper/212:20:212
2187499221,12cyclictest17727-21kthreadcore08:10:232
2187499221,11cyclictest6638-21latency_hist08:35:012
2186699228,5cyclictest0-21swapper/110:25:271
2186699225,11cyclictest0-21swapper/111:25:141
2186699223,3cyclictest131rcu_preempt10:10:151
2186699221,17cyclictest7820-21diskmemload12:00:131
2186699221,17cyclictest7820-21diskmemload12:00:131
2185499225,6cyclictest12-21ksoftirqd/009:40:250
2185499221,3cyclictest131rcu_preempt07:25:010
21854992211,7cyclictest30828-21apt-get09:00:140
2188299219,9cyclictest0-21swapper/310:10:363
2188299219,9cyclictest0-21swapper/309:33:353
2188299212,13cyclictest35966irq/58-eth011:20:153
2187499213,14cyclictest6419-21ssh09:45:142
2187499212,16cyclictest24315-21fschecks_time11:25:182
2187499212,10cyclictest6755-21sort08:35:142
2187499211,3cyclictest0-21swapper/210:00:172
2187499211,17cyclictest25867-21sed07:10:302
2187499211,16cyclictest16561-21fschecks_time08:45:192
2187499211,16cyclictest13096-21kthreadcore08:05:222
2187499211,11cyclictest15433-21kthreadcore07:35:212
2187499211,10cyclictest7644-21grep07:25:272
2187499211,10cyclictest21488-21kthreadcore07:40:212
2186699213,6cyclictest10434-21if_eth108:05:171
2186699212,13cyclictest21-21ksoftirqd/107:25:051
2186699211,9cyclictest5095-21cron08:00:001
2186699211,9cyclictest5095-21cron08:00:001
2186699211,7cyclictest19470-21cut08:45:261
2186699211,4cyclictest131rcu_preempt11:01:001
21866992113,4cyclictest2889-21processes10:35:291
2186699211,2cyclictest131rcu_preempt09:10:301
2186699211,16cyclictest11283-21apt-get08:40:041
2186699211,10cyclictest3563-21irqcore09:05:191
2186699210,6cyclictest171261/usr/sbin/munin10:45:141
2185499218,3cyclictest131rcu_preempt07:30:200
2185499217,4cyclictest12-21ksoftirqd/007:35:320
2185499216,4cyclictest12-21ksoftirqd/008:05:240
2185499216,3cyclictest131rcu_preempt07:20:010
2185499214,5cyclictest12-21ksoftirqd/008:10:190
2185499214,3cyclictest131rcu_preempt12:35:120
2185499211,6cyclictest53646usb-storage08:40:050
21854992112,3cyclictest131rcu_preempt08:50:300
21854992112,3cyclictest131rcu_preempt08:50:300
2188299209,8cyclictest0-21swapper/311:29:483
2188299203,12cyclictest0-21swapper/310:48:463
2188299203,11cyclictest35966irq/58-eth009:15:143
21882992010,6cyclictest0-21swapper/309:55:133
2187499207,4cyclictest9231-21sed07:30:182
2187499202,9cyclictest24264-21cut08:50:212
2187499202,9cyclictest24264-21cut08:50:212
2187499201,4cyclictest4535-21ssh10:20:122
2187499201,16cyclictest25339-21kernelversion08:20:182
2186699208,3cyclictest0-21swapper/111:33:511
2186699207,4cyclictest131rcu_preempt09:55:301
2186699205,4cyclictest131rcu_preempt12:15:021
2186699205,3cyclictest131rcu_preempt12:05:211
2186699202,9cyclictest131rcu_preempt10:24:231
2186699202,15cyclictest21468-21ssh12:15:341
21866992014,4cyclictest0-21swapper/111:10:151
2185499208,5cyclictest29496-21apt-get08:25:130
2185499204,3cyclictest131rcu_preempt08:30:130
2185499203,5cyclictest12-21ksoftirqd/008:20:260
2185499201,5cyclictest12-21ksoftirqd/008:00:290
2188299199,8cyclictest0-21swapper/312:06:553
2188299198,6cyclictest0-21swapper/309:51:543
2188299197,9cyclictest0-21swapper/310:20:143
2188299193,11cyclictest35966irq/58-eth009:35:133
2188299193,11cyclictest0-21swapper/311:14:463
2188299193,11cyclictest0-21swapper/310:44:463
2188299192,10cyclictest0-21swapper/307:20:133
2188299191,14cyclictest0-21swapper/311:45:173
2188299191,13cyclictest0-21swapper/309:00:303
2188299191,13cyclictest0-21swapper/308:55:293
2188299191,11cyclictest0-21swapper/308:15:013
2186699192,3cyclictest131rcu_preempt11:55:171
2186699192,2cyclictest131rcu_preempt08:35:171
2186699192,14cyclictest6163-21apt-get11:15:141
2186699191,3cyclictest0-21swapper/111:50:181
2186699191,2cyclictest131rcu_preempt09:27:251
2186699191,2cyclictest131rcu_preempt09:27:241
2186699190,6cyclictest20592-21kthreadcore07:40:211
2185499198,3cyclictest131rcu_preempt07:45:230
2185499192,5cyclictest12-21ksoftirqd/007:40:230
2188299189,6cyclictest0-21swapper/312:22:083
2188299189,6cyclictest0-21swapper/311:01:283
2188299188,8cyclictest0-21swapper/310:05:183
2188299183,10cyclictest0-21swapper/309:10:153
2188299182,10cyclictest0-21swapper/308:20:043
2188299182,10cyclictest0-21swapper/307:35:153
21882991812,4cyclictest0-21swapper/310:30:253
21882991810,6cyclictest0-21swapper/312:33:033
21866991813,3cyclictest0-21swapper/109:35:241
2186699181,2cyclictest131rcu_preempt08:20:011
2188299179,6cyclictest0-21swapper/312:35:123
2188299179,6cyclictest0-21swapper/307:54:363
2188299172,10cyclictest0-21swapper/309:05:153
2188299172,10cyclictest0-21swapper/308:15:123
2188299171,10cyclictest0-21swapper/308:40:043
2188299170,11cyclictest35966irq/58-eth008:50:133
2188299170,11cyclictest35966irq/58-eth008:50:133
2187499176,3cyclictest0-21swapper/207:10:022
2186699175,4cyclictest21-21ksoftirqd/108:20:031
21866991711,4cyclictest0-21swapper/108:50:281
21866991711,4cyclictest0-21swapper/108:50:271
2188299162,10cyclictest0-21swapper/308:25:143
2188299161,9cyclictest0-21swapper/308:35:163
2186699162,3cyclictest0-21swapper/108:55:231
2186699161,10cyclictest32146-21kthreadcore09:00:221
2188299152,9cyclictest0-21swapper/307:55:143
2188299152,9cyclictest0-21swapper/307:55:133
2186699152,4cyclictest31789-21grep07:20:151
2185499151,3cyclictest131rcu_preempt07:10:020
2188299142,9cyclictest0-21swapper/307:30:153
2188299142,10cyclictest0-21swapper/308:05:143
2186699141,3cyclictest131rcu_preempt07:10:021
2188299131,5cyclictest0-21swapper/307:45:203
2188299131,10cyclictest0-21swapper/307:10:283
2188299126,4cyclictest35966irq/58-eth007:25:043
2188299125,5cyclictest0-21swapper/308:05:013
2188299122,6cyclictest0-21swapper/308:45:163
2188299111,7cyclictest0-21swapper/308:30:183
2188299111,7cyclictest0-21swapper/307:15:213
218829991,6cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional