You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-10 - 17:02

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot4s.osadl.org (updated Sun May 10, 2026 12:45:12)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
15682996040,16cyclictest0-21swapper/311:20:123
1565399571,41cyclictest11523-21ssh11:20:110
1567499526,20cyclictest27-21ksoftirqd/208:55:142
1567499512,25cyclictest9067-21ssh10:25:132
1566699492,35cyclictest2005-21ssh09:10:111
1567499469,25cyclictest0-21swapper/209:30:132
15666994611,10cyclictest21-21ksoftirqd/109:25:131
1567499443,33cyclictest18147-21ssh10:30:152
1566699422,37cyclictest6135-21ssh11:35:141
1566699422,36cyclictest31676-21ssh12:25:111
1566699422,35cyclictest22693-21ssh12:20:131
1568299413,26cyclictest35966irq/58-eth011:15:133
15674994131,5cyclictest14120-21sh10:10:142
1566699411,18cyclictest24239-21ssh11:45:151
1568299402,26cyclictest35966irq/58-eth012:05:123
1566699402,31cyclictest17351-21ssh12:35:131
1565399402,30cyclictest4997-21ssh10:05:130
1568299393,22cyclictest35966irq/58-eth007:40:123
1566699392,33cyclictest898-21sh11:50:121
1566699392,32cyclictest15207-21ssh09:35:141
1566699391,4cyclictest131rcu_preempt12:05:111
1568299382,24cyclictest35966irq/58-eth008:05:123
1566699381,33cyclictest28285-21ssh10:00:121
1565399383,29cyclictest9933-21ssh11:55:120
1567499371,27cyclictest31087-21ssh10:55:142
1567499370,26cyclictest0-21swapper/211:45:142
1568299365,20cyclictest35966irq/58-eth011:35:143
1567499362,28cyclictest19242-21ssh09:55:142
1566699362,24cyclictest951-21ssh09:45:121
1566699361,26cyclictest25895-21ssh11:10:121
1566699361,26cyclictest25895-21ssh11:10:121
1566699361,25cyclictest27213-21rm10:35:151
1565399363,19cyclictest0-21swapper/009:45:120
1568299352,19cyclictest35966irq/58-eth011:45:133
15682993514,11cyclictest35966irq/58-eth008:40:123
1566699357,22cyclictest21-21ksoftirqd/107:25:131
1566699354,26cyclictest13623-21ssh12:15:131
1566699352,29cyclictest32412-21ssh10:20:121
1566699351,28cyclictest18943-21ssh12:00:101
1565399356,18cyclictest0-21swapper/010:25:130
1568299342,23cyclictest35966irq/58-eth007:45:123
1568299342,22cyclictest35966irq/58-eth009:55:143
1568299341,20cyclictest35966irq/58-eth011:05:143
1567499344,21cyclictest17322-21ssh12:35:122
1566699343,23cyclictest6097-21ssh09:30:131
15666993415,16cyclictest21-21ksoftirqd/110:45:121
1568299337,19cyclictest35966irq/58-eth010:35:143
1567499333,17cyclictest9942-21ssh11:55:132
15674993316,11cyclictest27-21ksoftirqd/211:50:132
1568299326,15cyclictest35966irq/58-eth009:20:133
1567499329,12cyclictest27-21ksoftirqd/212:25:142
15682993116,10cyclictest35966irq/58-eth012:15:123
15682993114,11cyclictest35966irq/58-eth009:45:123
1568299311,17cyclictest35966irq/58-eth010:55:163
1567499314,17cyclictest27346irq/40-dwc2_hso08:25:142
1567499314,17cyclictest27346irq/40-dwc2_hso08:25:142
1567499314,15cyclictest27346irq/40-dwc2_hso12:15:132
1566699318,16cyclictest21-21ksoftirqd/108:30:131
1567499309,15cyclictest6150-21ssh11:35:142
1567499305,18cyclictest20325-21ssh09:20:152
1567499304,19cyclictest27346irq/40-dwc2_hso08:15:142
1567499304,17cyclictest27346irq/40-dwc2_hso10:15:132
1567499302,8cyclictest28309-21ssh10:00:132
1566699301,2cyclictest0-21swapper/109:20:131
1568299292,16cyclictest35966irq/58-eth008:45:123
15682992918,7cyclictest0-21swapper/310:05:133
15682992914,8cyclictest35966irq/58-eth010:00:133
1567499296,13cyclictest27346irq/40-dwc2_hso08:40:122
1567499291,16cyclictest27231-21ssh10:35:152
15653992922,4cyclictest17176-21apt-get07:45:130
15653992914,10cyclictest24331-21apt-get09:40:140
15653992914,10cyclictest24331-21apt-get09:40:130
15653992913,12cyclictest6072-21apt-get09:30:130
1568299286,13cyclictest35966irq/58-eth011:10:133
1568299286,13cyclictest35966irq/58-eth011:10:123
15682992813,9cyclictest35966irq/58-eth011:30:253
1567499282,20cyclictest0-21swapper/207:20:132
1567499282,16cyclictest27346irq/40-dwc2_hso07:40:112
1567499282,11cyclictest0-21swapper/208:20:142
1567499281,19cyclictest2796-21cut07:30:152
15653992822,3cyclictest15039-21apt-get08:50:120
15653992822,3cyclictest15039-21apt-get08:50:120
1565399281,23cyclictest31086-21ssh10:55:140
1565399281,19cyclictest11200-21ssh09:15:130
1568299272,15cyclictest0-21swapper/311:55:133
1568299271,14cyclictest35966irq/58-eth008:10:123
1568299270,17cyclictest35966irq/58-eth012:20:123
1567499275,19cyclictest27346irq/40-dwc2_hso11:30:132
1567499272,15cyclictest4601-21ssh12:10:112
1566699272,19cyclictest21908-21ssh10:50:111
15653992713,10cyclictest620-21apt-get08:35:140
1567499263,15cyclictest27346irq/40-dwc2_hso10:40:132
1566699263,18cyclictest29604-21ssh11:30:141
1566699261,18cyclictest11199-21sh09:15:131
1565399262,15cyclictest2533-21ssh11:15:140
15653992615,9cyclictest27230-21ssh10:35:150
15653992615,7cyclictest0-21swapper/008:45:120
1568299252,17cyclictest0-21swapper/310:30:143
1568299252,15cyclictest0-21swapper/311:40:153
1568299252,15cyclictest0-21swapper/310:15:133
1567499254,16cyclictest0-21swapper/209:35:142
1567499253,16cyclictest27346irq/40-dwc2_hso11:15:132
1567499253,13cyclictest27-21ksoftirqd/207:10:262
1567499251,9cyclictest24334-21ssh09:40:122
1567499251,9cyclictest24334-21ssh09:40:122
1567499251,9cyclictest0-21swapper/208:10:132
15674992512,7cyclictest0-21swapper/210:50:122
1567499251,19cyclictest0-21swapper/207:55:132
1567499250,18cyclictest24672-21find09:00:132
1566699252,17cyclictest7739-21ssh11:00:131
1565399254,5cyclictest29138-21ssh11:10:230
1565399254,5cyclictest29138-21ssh11:10:220
1565399252,16cyclictest14113-21ssh10:10:140
1565399252,10cyclictest22433-21cat12:20:020
1565399251,9cyclictest5544-21cut08:40:150
15653992519,3cyclictest15865-21apt-get07:10:130
15653992512,11cyclictest27999-21apt-get12:05:110
1565399250,11cyclictest20839-21ssh09:36:250
15682992414,6cyclictest35966irq/58-eth009:35:153
1567499247,14cyclictest0-21swapper/211:20:112
1567499242,3cyclictest131rcu_preempt11:10:132
1567499242,3cyclictest131rcu_preempt11:10:132
1567499241,6cyclictest12854-21ssh10:45:122
1567499241,13cyclictest29402-21ssh09:25:132
1566699242,18cyclictest16844-21apt-get11:05:121
1566699242,17cyclictest15157-21ssh11:40:151
1565399244,4cyclictest12842-21apt-get10:45:130
1565399244,4cyclictest12-21ksoftirqd/012:25:130
1565399243,4cyclictest12-21ksoftirqd/009:13:200
1565399242,4cyclictest12-21ksoftirqd/009:28:120
1565399241,8cyclictest24534-21sed09:00:010
15653992418,3cyclictest23241-21apt-get10:15:130
15653992415,6cyclictest30285-21apt-get07:25:120
15653992413,4cyclictest0-21swapper/007:35:150
1568299239,10cyclictest321rcuc/307:30:123
1568299233,10cyclictest35966irq/58-eth008:55:143
1568299232,16cyclictest35966irq/58-eth009:05:153
1568299232,14cyclictest0-21swapper/312:35:133
1567499234,12cyclictest27346irq/40-dwc2_hso07:25:132
1567499231,13cyclictest8752-21grep12:30:182
1566699232,17cyclictest8356-21ssh12:30:151
1566699231,19cyclictest20748-21ls07:15:151
1565399236,9cyclictest0-21swapper/011:05:140
1565399236,14cyclictest0-21swapper/011:25:140
1565399234,4cyclictest3978-21ssh10:40:150
1565399234,4cyclictest18176-21ssh09:54:180
1565399233,12cyclictest0-21swapper/007:30:110
1565399232,5cyclictest12-21ksoftirqd/010:21:440
1565399232,17cyclictest15154-21diskmemload11:40:160
1565399231,7cyclictest4685-21/usr/sbin/munin12:10:120
15653992311,9cyclictest24681-21apt-get09:00:130
1565399230,9cyclictest18439-21grep12:35:180
1565399230,8cyclictest1039-21ssh11:50:150
1568299229,8cyclictest0-21swapper/311:50:133
1567499223,12cyclictest27346irq/40-dwc2_hso07:50:172
1567499223,12cyclictest27346irq/40-dwc2_hso07:50:162
1567499222,11cyclictest879-21unixbench_multi09:05:292
15674992213,6cyclictest23211-21ssh11:25:222
15674992210,10cyclictest3440-21ssh09:10:182
1566699222,13cyclictest2325-21ssh10:55:261
1566699221,18cyclictest5087-21ls10:05:151
1566699221,16cyclictest3942-21ssh10:40:141
1566699221,16cyclictest12301-21apt-get07:40:111
1566699220,4cyclictest19211-21apt-get09:55:141
1566699220,18cyclictest18659-21cut08:20:161
1565399229,8cyclictest12-21ksoftirqd/007:50:160
1565399229,8cyclictest12-21ksoftirqd/007:50:160
1565399225,5cyclictest0-21swapper/010:34:210
1565399225,11cyclictest19217-21ssh09:55:130
1565399224,6cyclictest25975-21ssh09:21:290
1565399222,7cyclictest21765-21rm10:50:030
1565399222,16cyclictest26793-21apt-get07:55:110
15653992214,5cyclictest3983-21apt-get08:05:130
15653992214,5cyclictest18935-21apt-get12:00:110
15653992212,7cyclictest24208-21apt-get11:45:120
15653992211,9cyclictest12-21ksoftirqd/011:00:120
15682992111,7cyclictest0-21swapper/311:00:213
1567499216,11cyclictest27346irq/40-dwc2_hso11:05:122
1567499212,5cyclictest0-21swapper/212:05:132
1567499212,15cyclictest28410-21tune2fs08:30:162
1567499212,11cyclictest6126-21kthreadcore08:05:212
1567499211,8cyclictest8151-21kthreadcore07:35:202
15674992114,4cyclictest930-21snmpd09:17:522
15674992111,8cyclictest0-21swapper/208:00:262
1567499211,16cyclictest10344-21fschecks_count08:45:162
1567499211,13cyclictest17396-21kthreadcore08:50:212
1567499211,13cyclictest17396-21kthreadcore08:50:202
1567499211,12cyclictest2968-21kthreadcore09:45:212
1567499211,12cyclictest18845-21sshd11:40:272
1567499211,11cyclictest24049-21cut12:20:192
1567499211,11cyclictest18910-21/usr/sbin/munin12:00:302
1567499211,10cyclictest1319-21kthreadcore10:20:192
1566699211,9cyclictest9156-21df10:25:161
1566699211,18cyclictest13926-21ls08:45:281
1566699211,16cyclictest4587-21apt-get12:10:101
1566699211,14cyclictest1651-21ssh09:10:011
1566699211,14cyclictest14283-21cut10:10:161
1565399218,9cyclictest15831-21/usr/sbin/munin07:40:260
1565399216,8cyclictest0-21swapper/011:37:530
1565399216,12cyclictest379-21needreboot09:05:230
1565399216,11cyclictest21912-21ntp_192.168.11408:20:240
15653992116,3cyclictest7928-21apt-get12:30:020
1568299204,9cyclictest35966irq/58-eth009:30:133
1568299202,12cyclictest35966irq/58-eth009:10:113
1568299201,14cyclictest0-21swapper/310:10:143
1567499205,11cyclictest7709-21apt-get11:00:122
1567499203,9cyclictest6243-21kthreadcore10:05:202
1567499202,15cyclictest3888-21latency08:35:232
1567499201,10cyclictest11892-21kthreadcore09:50:192
1567499200,11cyclictest17331-21fschecks_time07:45:152
1566699203,10cyclictest0-21swapper/107:45:131
1565399206,9cyclictest25470-21kthreadcore12:20:210
1565399204,3cyclictest0-21swapper/008:25:230
1565399204,3cyclictest0-21swapper/008:25:220
1565399204,12cyclictest0-21swapper/007:15:120
1565399202,14cyclictest28812-21ssh10:00:190
1565399201,15cyclictest29575-21apt-get11:30:140
1568299199,8cyclictest0-21swapper/312:14:163
1568299199,8cyclictest0-21swapper/310:47:383
1568299198,8cyclictest0-21swapper/312:00:253
1568299192,11cyclictest0-21swapper/308:30:043
15682991913,4cyclictest35966irq/58-eth009:15:143
15682991912,4cyclictest0-21swapper/310:40:313
15682991911,5cyclictest0-21swapper/309:40:133
15682991911,5cyclictest0-21swapper/309:40:123
1568299191,12cyclictest0-21swapper/308:20:173
1568299191,12cyclictest0-21swapper/307:20:173
15682991910,7cyclictest0-21swapper/309:53:513
15682991910,6cyclictest0-21swapper/310:25:153
1567499191,9cyclictest20849-21fschecks_time07:15:152
1566699195,9cyclictest0-21swapper/108:00:121
1566699195,11cyclictest5793-21irqcore08:40:181
1566699194,10cyclictest15647-21cyclictest08:15:141
1566699191,9cyclictest20031-21fschecks_time08:55:171
1566699191,13cyclictest10038-21df_inode11:55:151
15666991910,7cyclictest21903-21open_inodes10:30:281
15666991910,6cyclictest15754-21cat11:20:281
1566699190,13cyclictest19607-21timerwakeupswit07:10:271
1565399197,9cyclictest0-21swapper/008:00:110
1565399196,4cyclictest0-21swapper/008:10:160
1565399194,4cyclictest13714-21apt-get08:15:140
1568299189,6cyclictest0-21swapper/310:22:273
1568299183,8cyclictest0-21swapper/307:15:123
1568299183,10cyclictest0-21swapper/312:25:183
1568299182,10cyclictest0-21swapper/309:00:143
1568299182,10cyclictest0-21swapper/308:50:123
1568299182,10cyclictest0-21swapper/308:50:123
1568299182,10cyclictest0-21swapper/308:25:053
1568299182,10cyclictest0-21swapper/308:25:043
1568299182,10cyclictest0-21swapper/308:00:133
1568299182,10cyclictest0-21swapper/307:35:153
15682991810,6cyclictest0-21swapper/309:26:193
15682991810,4cyclictest0-21swapper/312:32:293
1566699181,3cyclictest0-21swapper/108:00:011
1566699181,14cyclictest7305-21grep08:05:221
1566699181,14cyclictest12873-21kthreadcore09:50:201
1566699181,13cyclictest26850-21sshd08:25:261
1566699181,13cyclictest26850-21sshd08:25:261
1566699181,13cyclictest23380-21ssh10:15:141
1566699181,13cyclictest13406-21munin-run08:15:021
15666991810,5cyclictest4516-21kthreadcore11:15:231
15666991810,5cyclictest20794-21fschecks_time11:25:171
1565399183,4cyclictest12-21ksoftirqd/007:20:270
15653991812,4cyclictest0-21swapper/008:30:130
1568299179,6cyclictest0-21swapper/310:50:073
1568299172,11cyclictest0-21swapper/311:25:143
1568299172,10cyclictest0-21swapper/308:15:143
1568299171,11cyclictest0-21swapper/307:55:273
1566699177,7cyclictest8167-21kthreadcore07:35:201
1566699175,9cyclictest15744-21kthreadcore08:50:181
1566699175,9cyclictest15744-21kthreadcore08:50:171
1566699172,10cyclictest5910-21kthreadcore07:30:211
1566699171,7cyclictest789-21fschecks_time08:35:171
1566699171,13cyclictest27607-21kthreadcore07:20:201
1566699171,13cyclictest24353-21ssh09:40:131
1566699171,13cyclictest24353-21ssh09:40:131
1568299162,9cyclictest0-21swapper/307:10:133
1568299161,8cyclictest35966irq/58-eth007:50:153
1568299161,8cyclictest35966irq/58-eth007:50:153
1566699161,11cyclictest25396-21ntp_192.168.11407:50:281
1566699161,11cyclictest25396-21ntp_192.168.11407:50:271
1566699156,6cyclictest25081-21irqcore09:00:181
1565399153,9cyclictest15733-21latency_hist07:10:020
1567499146,5cyclictest15756-21latency_hist07:10:022
1566699141,10cyclictest15734-21cat07:10:031
1568299131,5cyclictest0-21swapper/307:10:023
1568299131,10cyclictest0-21swapper/308:35:203
1568299121,8cyclictest0-21swapper/307:25:143
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional