You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-21 - 01:31

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Sat Jun 20, 2026 12:45:10)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2802299743,55cyclictest11851-21ssh11:15:111
2800999701,6cyclictest131rcu_preempt12:00:120
2800999664,6cyclictest131rcu_preempt08:15:120
28022996316,9cyclictest131rcu_preempt12:05:151
28022996024,26cyclictest21-21ksoftirqd/108:40:151
2802299581,50cyclictest13802-21ssh10:40:131
2800999572,38cyclictest1233-21ssh10:15:130
2804799537,29cyclictest35966irq/58-eth011:30:143
2802299501,36cyclictest3303-21ssh09:40:161
2804799484,27cyclictest35966irq/58-eth011:50:153
2802299432,35cyclictest32238-21ssh09:20:131
2800999431,27cyclictest8900-21ssh09:25:150
2802299423,28cyclictest17347-21ssh11:00:121
28009994210,4cyclictest131rcu_preempt11:55:270
28037994113,10cyclictest27-21ksoftirqd/212:05:142
2802299414,30cyclictest6685-21ssh10:00:131
2800999412,33cyclictest28199-21ssh10:30:140
2800999412,33cyclictest28199-21ssh10:30:130
28037994011,24cyclictest27-21ksoftirqd/211:15:172
2802299401,27cyclictest9842-21ssh11:50:141
28009993913,17cyclictest12-21ksoftirqd/010:50:130
28009993913,17cyclictest12-21ksoftirqd/010:50:130
2803799381,29cyclictest26318-21ssh11:05:122
2800999384,27cyclictest30115-21sh09:55:130
2804799365,21cyclictest35966irq/58-eth009:15:143
28047993624,6cyclictest35966irq/58-eth011:05:133
2804799362,22cyclictest35966irq/58-eth009:20:123
2804799361,20cyclictest35966irq/58-eth011:55:133
28037993618,13cyclictest0-21swapper/211:39:172
2803799352,25cyclictest14214-21sed10:40:172
28047993414,10cyclictest35966irq/58-eth011:35:133
2803799342,24cyclictest23486-21irqstats10:45:192
28037993413,15cyclictest0-21swapper/208:25:142
28047993314,12cyclictest0-21swapper/311:40:143
2800999332,26cyclictest25673-21ssh12:35:130
2804799325,18cyclictest35966irq/58-eth008:25:163
2804799322,21cyclictest35966irq/58-eth012:05:143
2804799322,19cyclictest35966irq/58-eth009:50:123
2803799322,21cyclictest28680-21cut10:30:172
2803799322,21cyclictest28680-21cut10:30:172
2803799322,19cyclictest27346irq/40-dwc2_hso07:10:142
2802299322,22cyclictest6354-21ssh11:30:141
2800999323,26cyclictest21219-21ssh09:50:130
2800999322,26cyclictest31235-21ssh12:20:130
2804799312,17cyclictest35966irq/58-eth009:30:143
2802299313,18cyclictest21-21ksoftirqd/109:15:141
2802299312,21cyclictest3389-21cut11:10:171
2800999312,23cyclictest22448-21ssh11:38:170
28009993111,7cyclictest12-21ksoftirqd/008:45:130
2800999302,22cyclictest6596-21hostname11:30:170
2800999302,20cyclictest14246-21ssh09:10:130
2803799293,17cyclictest27346irq/40-dwc2_hso11:30:132
2803799292,17cyclictest27346irq/40-dwc2_hso07:15:132
2800999292,21cyclictest11168-21sh10:37:160
2800999291,22cyclictest8590-21fw_packets10:55:170
2800999282,20cyclictest25347-21ssh11:04:170
2800999282,20cyclictest20948-21ssh10:43:170
2800999282,18cyclictest22209-21ssh12:15:130
2800999281,18cyclictest16704-21ssh12:30:120
2804799273,14cyclictest35966irq/58-eth008:05:133
28037992713,4cyclictest27-21ksoftirqd/212:00:122
2800999272,19cyclictest13151-21ssh12:10:140
2800999272,18cyclictest26688-21if_eth111:05:170
2803799263,18cyclictest0-21swapper/211:10:172
28037992615,7cyclictest261rcuc/209:20:132
28037992615,3cyclictest0-21swapper/212:10:132
2800999263,18cyclictest10252-21ssh10:20:140
2804799251,16cyclictest0-21swapper/309:35:133
2804799251,12cyclictest35966irq/58-eth010:50:133
2804799251,12cyclictest35966irq/58-eth010:50:133
2803799257,8cyclictest17488-21cron09:30:012
2803799253,17cyclictest0-21swapper/211:00:382
2803799252,9cyclictest22599-21ssh12:15:172
2803799252,8cyclictest21240-21ssh11:20:172
28037992513,9cyclictest26165-21ssh09:34:482
28037992510,4cyclictest27-21ksoftirqd/209:56:592
2802299253,16cyclictest10225-21apt-get10:20:121
2804799247,11cyclictest35966irq/58-eth010:35:143
2804799243,10cyclictest0-21swapper/310:43:173
2804799242,14cyclictest0-21swapper/310:30:123
2804799242,14cyclictest0-21swapper/310:30:123
2803799241,17cyclictest9857-21ssh11:50:152
2800999249,11cyclictest0-21swapper/010:05:130
2800999243,14cyclictest7753-21sh12:25:110
2800999241,17cyclictest2893-21ssh11:10:140
2800999241,17cyclictest23265-21ssh09:15:130
2804799231,12cyclictest35966irq/58-eth010:15:143
2803799239,10cyclictest0-21swapper/211:45:232
2803799239,10cyclictest0-21swapper/211:45:222
2803799237,14cyclictest15213-21cron10:05:002
2803799234,12cyclictest27346irq/40-dwc2_hso08:40:132
2803799232,6cyclictest0-21swapper/210:27:172
2803799232,16cyclictest0-21swapper/210:58:172
2803799232,16cyclictest0-21swapper/210:50:372
2803799232,16cyclictest0-21swapper/210:50:362
2803799232,16cyclictest0-21swapper/210:39:182
28037992313,8cyclictest0-21swapper/208:00:342
28037992312,8cyclictest10409-21grep10:20:162
28022992314,6cyclictest0-21swapper/111:20:171
28022992313,4cyclictest0-21swapper/111:40:251
2802299231,18cyclictest4295-21ls09:00:011
2804799223,13cyclictest0-21swapper/311:10:173
2804799223,12cyclictest0-21swapper/312:10:133
28047992213,5cyclictest0-21swapper/312:20:133
2803799223,11cyclictest12876-21fschecks_count08:35:162
2803799222,16cyclictest709-21rs:main2
28037992217,2cyclictest27-21ksoftirqd/208:30:132
28037992213,7cyclictest0-21swapper/212:30:142
28037992212,8cyclictest0-21swapper/212:20:232
28037992212,4cyclictest0-21swapper/212:35:332
2803799221,17cyclictest7730-21find12:25:112
2803799221,12cyclictest19269-21sed11:55:192
2803799221,11cyclictest930-21snmpd09:12:502
2802299229,8cyclictest21-21ksoftirqd/110:30:321
2802299229,8cyclictest21-21ksoftirqd/110:30:311
2802299224,13cyclictest21-21ksoftirqd/110:25:311
2802299221,17cyclictest15729-21ssh10:05:151
2802299221,14cyclictest26760-21ssh09:35:141
2800999222,16cyclictest20921-21ssh11:20:130
2800999222,14cyclictest29828-21ssh11:25:110
28009992213,6cyclictest25589-21kthreadcore10:45:210
2800999221,17cyclictest17576-21apt-get08:40:120
2804799214,12cyclictest0-21swapper/312:15:133
2804799212,13cyclictest0-21swapper/312:30:133
2804799211,15cyclictest0-21swapper/309:25:303
28047992110,8cyclictest0-21swapper/310:21:493
2803799219,9cyclictest30427-21ssh10:11:352
2803799219,9cyclictest30243-21ssh11:41:442
2803799216,7cyclictest87766lldpd08:47:262
2803799213,9cyclictest6498-21kthreadcore09:00:202
2803799212,10cyclictest5624-21fschecks_time07:20:162
2803799211,9cyclictest14808-21apt07:30:012
28037992111,8cyclictest0-21swapper/209:47:182
2803799211,16cyclictest32509-21kthreadcore07:45:222
2803799211,11cyclictest4087-21cut09:40:212
2803799211,10cyclictest6477-21cron07:55:002
2803799211,10cyclictest31270-21usage_EthcTask09:35:312
2803799211,10cyclictest30065-21fschecks_time11:25:152
2803799211,10cyclictest19817-21proc_pri08:05:272
28037992110,9cyclictest0-21swapper/210:15:252
2802299219,3cyclictest21-21ksoftirqd/111:47:491
2802299219,3cyclictest21-21ksoftirqd/111:47:481
2802299212,9cyclictest0-21swapper/112:23:411
2802299211,14cyclictest21239-21grep08:10:171
2802299210,16cyclictest32650-21ssh11:07:181
2802299210,15cyclictest21184-21find09:50:131
2800999213,9cyclictest12-21ksoftirqd/010:15:020
2800999211,6cyclictest10378-21/usr/sbin/munin10:00:270
28009992112,6cyclictest0-21swapper/009:45:100
2800999211,16cyclictest1907-21find07:50:120
28009992110,8cyclictest0-21swapper/009:30:200
2804799209,8cyclictest0-21swapper/310:10:313
2804799203,12cyclictest0-21swapper/311:15:173
28047992011,4cyclictest0-21swapper/312:35:313
2803799203,9cyclictest32098-21fschecks_count08:55:172
2803799203,15cyclictest13791-21apt-key09:10:032
2803799203,14cyclictest22795-21kthreadcore08:10:222
2803799202,9cyclictest834-21kthreadcore08:20:202
2803799201,9cyclictest26225-21sort08:15:172
2803799201,9cyclictest24950-21if_eth107:40:182
2803799201,14cyclictest0-21swapper/207:35:142
2803799201,10cyclictest27646-21cut08:50:182
2803799201,10cyclictest27302-21grep09:15:292
2803799201,10cyclictest17592-21grep10:05:222
2802299203,3cyclictest131rcu_preempt12:15:101
2802299202,14cyclictest6718-21apt-get07:55:131
2802299202,14cyclictest11843-21memory10:55:241
2802299201,9cyclictest131rcu_preempt09:25:031
2802299201,3cyclictest131rcu_preempt08:50:191
28022992013,4cyclictest1203-21mailstats11:25:261
28022992012,6cyclictest14228-21apt09:10:121
2802299201,10cyclictest131rcu_preempt09:34:481
28022992010,7cyclictest20603-21find12:30:291
2802299200,3cyclictest131rcu_preempt09:45:101
2802299200,15cyclictest22781-21ssh10:45:121
2802299200,14cyclictest9998-21latency_hist07:25:011
2800999208,9cyclictest13759-21cron09:10:010
2800999207,6cyclictest27996-21cyclictest11:40:150
28009992013,4cyclictest0-21swapper/008:10:270
2804799198,8cyclictest0-21swapper/310:45:593
2804799198,8cyclictest0-21swapper/309:10:153
2804799193,11cyclictest0-21swapper/311:00:383
2804799193,11cyclictest0-21swapper/310:55:363
2804799192,11cyclictest0-21swapper/308:15:133
2804799192,10cyclictest0-21swapper/308:50:143
2804799191,13cyclictest0-21swapper/307:50:153
2804799191,13cyclictest0-21swapper/307:40:163
2804799191,12cyclictest0-21swapper/309:50:013
28047991910,7cyclictest0-21swapper/311:45:173
28047991910,7cyclictest0-21swapper/311:45:173
28047991910,6cyclictest35966irq/58-eth011:25:133
2803799192,9cyclictest6889-21fschecks_time07:55:162
2803799192,9cyclictest15141-21grep07:30:152
2802299196,8cyclictest21-21ksoftirqd/108:30:251
2802299192,3cyclictest131rcu_preempt12:10:091
2802299191,4cyclictest131rcu_preempt12:38:001
28022991913,4cyclictest0-21swapper/107:10:291
2802299191,14cyclictest12732-21apt-get08:35:131
2802299191,11cyclictest6481-21latency_hist07:55:011
28022991910,5cyclictest21-21ksoftirqd/111:35:131
2802299190,3cyclictest131rcu_preempt10:51:211
2802299190,3cyclictest131rcu_preempt10:51:201
2800999199,7cyclictest17406-21sshd11:16:170
2800999196,8cyclictest0-21swapper/007:55:120
2800999196,3cyclictest131rcu_preempt12:05:170
2800999191,13cyclictest1-21systemd08:35:150
2800999190,16cyclictest11223-21ls08:30:210
2804799189,6cyclictest0-21swapper/309:40:513
2804799187,8cyclictest0-21swapper/312:01:583
2804799183,11cyclictest0-21swapper/310:29:173
2804799183,10cyclictest0-21swapper/311:20:433
2804799182,11cyclictest0-21swapper/307:25:143
2804799182,10cyclictest0-21swapper/309:00:043
2804799182,10cyclictest0-21swapper/308:35:143
2804799182,10cyclictest0-21swapper/308:30:133
2804799182,10cyclictest0-21swapper/307:55:133
2804799182,10cyclictest0-21swapper/307:15:133
2804799181,13cyclictest0-21swapper/307:35:013
2802299183,3cyclictest131rcu_preempt10:10:581
2802299181,9cyclictest26007-21fschecks_count08:15:151
2802299181,4cyclictest131rcu_preempt11:56:371
2802299181,2cyclictest131rcu_preempt09:55:121
28022991812,4cyclictest0-21swapper/109:00:281
28022991811,4cyclictest27962-21gpgv07:10:011
28022991811,4cyclictest27962-21gpgv07:10:011
2802299181,14cyclictest1340-21ssh10:15:151
2800999189,6cyclictest0-21swapper/008:20:110
2800999187,3cyclictest12-21ksoftirqd/009:20:260
2800999182,11cyclictest0-21swapper/008:00:150
2800999181,14cyclictest26759-21sh09:35:150
2800999181,13cyclictest19191-21sh10:25:140
2800999181,11cyclictest4607-21grep09:00:160
2804799179,6cyclictest0-21swapper/312:25:453
2804799179,6cyclictest0-21swapper/310:05:083
2804799179,6cyclictest0-21swapper/310:01:553
2804799179,6cyclictest0-21swapper/309:56:333
2804799172,9cyclictest0-21swapper/308:20:133
2802299177,2cyclictest131rcu_preempt07:15:131
2802299176,2cyclictest131rcu_preempt10:35:021
2802299174,11cyclictest31174-21kthreadcore07:45:221
2802299173,3cyclictest131rcu_preempt07:35:171
2802299171,8cyclictest9883-21kthreadcore09:05:201
2802299171,3cyclictest27691-21apt-get12:00:111
2802299171,3cyclictest131rcu_preempt12:25:261
2802299171,3cyclictest131rcu_preempt08:00:211
2802299171,2cyclictest131rcu_preempt08:45:241
28022991711,4cyclictest7728-21sh08:30:011
2802299171,11cyclictest13575-21missed_timers07:25:231
2802299170,11cyclictest16317-21acpi08:05:121
2800999179,6cyclictest31713-21cron08:55:030
2800999179,5cyclictest10159-21latency_hist07:25:030
2800999173,11cyclictest17497-21tr08:05:200
2800999172,11cyclictest26779-21kthreadcore07:40:220
2800999172,10cyclictest0-21swapper/007:45:140
2800999171,3cyclictest131rcu_preempt11:45:100
2800999171,3cyclictest131rcu_preempt11:45:100
2800999171,13cyclictest15216-21cut07:30:170
2800999171,13cyclictest13242-21kthreadcore07:25:220
2800999171,12cyclictest6996-21tail08:25:310
2800999171,12cyclictest4341-21tr09:40:210
2800999171,12cyclictest0-21swapper/011:50:130
2804799162,9cyclictest0-21swapper/307:10:053
2804799162,10cyclictest0-21swapper/309:05:143
2804799162,10cyclictest0-21swapper/308:00:153
2804799161,12cyclictest0-21swapper/307:35:293
2803799164,9cyclictest28167-21gpgv07:10:012
2803799164,9cyclictest28167-21gpgv07:10:012
2802299168,6cyclictest30763-21apt-get08:20:111
2802299166,8cyclictest24645-21apt-get07:40:131
2802299161,3cyclictest131rcu_preempt07:30:171
2800999168,5cyclictest1115-21irqrtprio07:15:180
2800999166,7cyclictest1-21systemd08:50:190
2800999162,11cyclictest29157-21kthreadcore07:10:200
2800999161,12cyclictest22019-21tr07:35:220
2804799159,4cyclictest0-21swapper/308:45:163
2804799152,10cyclictest0-21swapper/308:55:143
2804799151,10cyclictest0-21swapper/308:40:293
2800999151,12cyclictest27996-21cyclictest07:10:010
2800999151,12cyclictest27996-21cyclictest07:10:010
2804799122,7cyclictest0-21swapper/307:45:173
2804799122,7cyclictest0-21swapper/307:10:023
2804799122,7cyclictest0-21swapper/307:10:013
2804799121,9cyclictest0-21swapper/308:10:213
2804799111,8cyclictest0-21swapper/307:20:193
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional