You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-10 - 20:33

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Fri Jul 10, 2026 12:45:15)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
362999672,41cyclictest5102-21ssh10:30:122
362999591,36cyclictest20689-21ssh09:45:112
362999591,34cyclictest19466-21ssh10:20:132
360899533,47cyclictest30786-21ssh11:20:120
362999502,33cyclictest15446-21ssh10:00:112
362999502,33cyclictest15446-21ssh10:00:112
362999451,28cyclictest18836-21ssh12:25:152
363799417,24cyclictest35966irq/58-eth010:40:113
363799417,24cyclictest35966irq/58-eth010:40:113
360899411,27cyclictest11227-21ssh11:45:110
363799405,23cyclictest35966irq/58-eth007:25:153
363799405,23cyclictest35966irq/58-eth007:25:143
362999405,17cyclictest27-21ksoftirqd/209:40:132
362999403,22cyclictest8983-21ssh10:50:122
363799397,20cyclictest35966irq/58-eth012:25:143
363799392,24cyclictest35966irq/58-eth007:45:153
3637993822,11cyclictest35966irq/58-eth012:10:113
363799371,19cyclictest35966irq/58-eth011:00:123
3637993619,11cyclictest35966irq/58-eth009:40:133
363799352,21cyclictest35966irq/58-eth012:30:113
3637993512,11cyclictest35966irq/58-eth011:55:143
360899356,18cyclictest12-21ksoftirqd/010:55:140
3629993427,3cyclictest23242-21ssh10:40:122
3629993427,3cyclictest23242-21ssh10:40:122
363799335,19cyclictest321rcuc/309:10:133
363799326,17cyclictest33-21ksoftirqd/310:05:123
3637993221,8cyclictest0-21swapper/310:35:133
3629993215,9cyclictest0-21swapper/208:55:132
362999315,17cyclictest27346irq/40-dwc2_hso08:25:152
362999311,24cyclictest16523-21ssh11:30:122
362999303,19cyclictest27346irq/40-dwc2_hso12:35:122
362999303,18cyclictest27346irq/40-dwc2_hso08:20:142
362999303,18cyclictest27346irq/40-dwc2_hso08:20:142
362999302,19cyclictest27346irq/40-dwc2_hso11:25:132
362999302,15cyclictest30805-21ssh11:20:132
363799296,12cyclictest35966irq/58-eth010:15:123
363799294,14cyclictest35966irq/58-eth009:50:133
362999292,20cyclictest0-21swapper/207:10:132
362999291,19cyclictest27346irq/40-dwc2_hso12:00:122
362999291,12cyclictest261rcuc/211:50:122
362999291,10cyclictest27-21ksoftirqd/208:45:132
362999288,8cyclictest24160-21ssh12:10:112
362999282,20cyclictest0-21swapper/209:00:152
362999281,21cyclictest1154-21ssh10:10:142
3629992812,12cyclictest22135-21sh09:10:152
362999281,17cyclictest27346irq/40-dwc2_hso12:30:112
361999285,14cyclictest21710-21diskmemload12:05:131
361999282,19cyclictest32378-21ssh10:45:141
361999281,5cyclictest16496-21apt-get11:30:121
3619992812,10cyclictest0-21swapper/108:00:121
360899281,2cyclictest131rcu_preempt07:45:140
363799273,18cyclictest0-21swapper/312:39:153
362999279,12cyclictest27-21ksoftirqd/210:35:122
362999273,13cyclictest12721-21ssh11:10:132
3629992711,13cyclictest32350-21apt-get10:45:122
361999272,19cyclictest854-21sshd12:38:251
3637992613,6cyclictest0-21swapper/308:00:123
363799261,13cyclictest35966irq/58-eth011:50:123
3637992610,10cyclictest35966irq/58-eth011:40:143
362999263,9cyclictest261rcuc/212:20:112
362999262,20cyclictest0-21swapper/207:55:152
362999262,20cyclictest0-21swapper/207:55:152
362999261,12cyclictest10253-21ssh10:15:132
360899264,5cyclictest12-21ksoftirqd/011:25:250
360899262,18cyclictest25911-21ssh09:30:130
363799254,13cyclictest35966irq/58-eth008:40:133
363799252,17cyclictest321rcuc/307:10:123
363799252,13cyclictest35966irq/58-eth011:25:123
363799251,13cyclictest35966irq/58-eth009:05:133
362999257,11cyclictest0-21swapper/211:00:112
362999252,14cyclictest27346irq/40-dwc2_hso08:40:122
362999251,19cyclictest27346irq/40-dwc2_hso11:35:122
361999251,17cyclictest24347-21ssh09:45:241
360899251,18cyclictest28459-21ssh10:25:120
3637992413,4cyclictest0-21swapper/312:00:113
363799241,14cyclictest35966irq/58-eth010:00:103
363799241,14cyclictest35966irq/58-eth010:00:103
362999243,15cyclictest27346irq/40-dwc2_hso08:50:122
3629992412,9cyclictest8590-21ssh11:42:202
362999241,19cyclictest761-21ssh12:15:142
361999246,10cyclictest3596-21cyclictest11:20:121
3619992410,4cyclictest131rcu_preempt11:40:021
360899242,16cyclictest9126-21ssh10:50:150
360899241,19cyclictest14544-21apt-get07:55:140
360899241,19cyclictest14544-21apt-get07:55:140
362999239,11cyclictest19816-21grep08:30:312
362999238,5cyclictest30617-21kthreadcore09:50:202
362999233,17cyclictest27346irq/40-dwc2_hso09:15:122
362999233,13cyclictest27346irq/40-dwc2_hso11:05:112
362999232,6cyclictest0-21swapper/208:15:042
362999231,8cyclictest19439-21/usr/sbin/munin08:00:192
362999231,12cyclictest7813-21ssh09:20:142
362999231,11cyclictest11255-21ssh11:45:122
361999239,3cyclictest131rcu_preempt12:25:241
360899235,2cyclictest131rcu_preempt09:00:290
360899234,11cyclictest12-21ksoftirqd/010:40:130
360899234,11cyclictest12-21ksoftirqd/010:40:130
360899232,18cyclictest29492-21ssh11:55:140
3608992318,3cyclictest536-21usb-storage10:45:050
3608992314,5cyclictest0-21swapper/009:25:150
363799223,12cyclictest0-21swapper/310:50:143
362999223,16cyclictest1211-21idleruntime-cro08:15:012
362999221,14cyclictest27346irq/40-dwc2_hso07:20:122
362999221,11cyclictest9500-21cron07:50:012
3629992210,8cyclictest15553-21interrupts12:05:182
361999221,15cyclictest18275-21cut10:55:181
360899225,3cyclictest131rcu_preempt12:25:230
3608992210,3cyclictest21710-21diskmemload10:03:540
3608992210,3cyclictest21710-21diskmemload10:03:540
363799215,11cyclictest321rcuc/309:35:123
363799212,10cyclictest0-21swapper/312:05:123
363799211,12cyclictest35966irq/58-eth007:55:143
363799211,12cyclictest35966irq/58-eth007:55:143
362999216,6cyclictest14334-21latency_hist07:55:012
362999214,9cyclictest25880-21apt-get09:30:112
362999214,11cyclictest0-21swapper/208:35:142
3629992115,4cyclictest0-21swapper/210:05:122
3629992112,7cyclictest32215-21kthreadcore11:55:222
362999211,11cyclictest9787-21kthreadcore09:55:232
362999211,10cyclictest7259-21usage_EthcTask09:35:322
361999217,10cyclictest28594-21grep10:05:291
361999212,15cyclictest32532-21apt-get07:40:131
361999211,16cyclictest24270-21apt-get08:05:131
361999211,16cyclictest24270-21apt-get08:05:131
360899213,3cyclictest131rcu_preempt11:00:160
360899211,14cyclictest17776-21ssh10:35:240
360899211,11cyclictest5822-21irqstats10:30:180
3608992110,9cyclictest31151-21apt-get09:15:120
363799203,13cyclictest0-21swapper/312:20:133
363799203,10cyclictest321rcuc/311:10:133
3637992014,4cyclictest0-21swapper/311:35:293
3637992014,4cyclictest0-21swapper/309:20:383
3637992014,2cyclictest0-21swapper/311:20:103
3637992012,5cyclictest0-21swapper/312:15:293
3637992011,4cyclictest0-21swapper/310:22:433
3637992010,8cyclictest35966irq/58-eth011:15:133
3637992010,7cyclictest35966irq/58-eth010:25:133
362999203,14cyclictest18869-21kthreadcore09:25:232
362999202,15cyclictest27863-21fschecks_count07:35:152
362999201,9cyclictest2285-21kthreadcore07:40:212
362999201,9cyclictest18207-21fschecks_count10:55:172
362999201,9cyclictest10913-21kthreadcore07:15:222
362999201,15cyclictest29560-21kthreadcore10:25:192
362999201,10cyclictest25217-21wc11:15:232
362999201,10cyclictest17906-21kthreadcore09:05:192
362999200,15cyclictest24418-21tune2fs08:05:182
362999200,15cyclictest24418-21tune2fs08:05:182
361999206,6cyclictest26247-21ssh09:10:311
361999205,9cyclictest16876-21ssh09:25:151
3619992013,5cyclictest0-21swapper/111:15:161
3619992010,7cyclictest21-21ksoftirqd/110:50:121
360899209,6cyclictest0-21swapper/012:39:150
360899206,3cyclictest21710-21diskmemload10:15:020
360899203,11cyclictest2225-21ssh11:40:140
360899203,11cyclictest0-21swapper/012:05:120
360899201,9cyclictest0-21swapper/012:10:050
360899201,7cyclictest20690-21apt-get09:45:130
360899201,11cyclictest21710-21diskmemload10:23:110
3608992010,3cyclictest21710-21diskmemload11:18:080
363799194,8cyclictest35966irq/58-eth011:45:123
363799194,10cyclictest321rcuc/308:35:123
363799194,10cyclictest0-21swapper/310:45:053
3637991913,4cyclictest0-21swapper/311:30:123
3637991913,4cyclictest0-21swapper/311:05:213
3637991913,4cyclictest0-21swapper/309:58:553
3637991913,4cyclictest0-21swapper/309:20:013
363799191,12cyclictest0-21swapper/307:45:013
363799191,11cyclictest0-21swapper/308:30:013
363799191,11cyclictest0-21swapper/308:15:163
3637991910,6cyclictest0-21swapper/310:55:223
3637991910,4cyclictest0-21swapper/309:25:163
362999192,8cyclictest23976-21kthreadcore07:30:222
362999191,9cyclictest18256-21fschecks_count07:25:182
362999191,9cyclictest18256-21fschecks_count07:25:182
361999191,9cyclictest1654-21find08:45:291
3619991912,4cyclictest30483-21kthreadcore10:25:211
361999190,15cyclictest11587-21find09:40:111
360899199,8cyclictest9736-21apt-get07:50:130
360899197,9cyclictest0-21swapper/008:15:180
360899191,9cyclictest0-21swapper/009:59:480
360899191,8cyclictest16766systemd-network11:51:580
3608991914,3cyclictest0-21swapper/007:30:150
3608991912,4cyclictest0-21swapper/009:20:280
3608991910,7cyclictest9788-21apt-get12:20:120
3608991910,3cyclictest21710-21diskmemload11:39:540
363799188,8cyclictest0-21swapper/309:34:193
363799182,10cyclictest0-21swapper/308:55:143
363799182,10cyclictest0-21swapper/307:20:133
363799181,12cyclictest0-21swapper/310:10:263
363799181,12cyclictest0-21swapper/308:05:303
363799181,12cyclictest0-21swapper/308:05:303
363799181,12cyclictest0-21swapper/307:30:233
362999181,8cyclictest3647-21latency_hist07:10:022
361999184,9cyclictest0-21swapper/109:55:121
361999184,11cyclictest31413-21tr11:55:201
361999183,8cyclictest0-21swapper/111:40:131
361999182,11cyclictest0-21swapper/109:35:141
361999181,9cyclictest6530-21fschecks_time08:20:171
361999181,9cyclictest6530-21fschecks_time08:20:171
361999181,7cyclictest2363-21ntpq09:15:251
361999181,3cyclictest0-21swapper/109:30:131
361999181,10cyclictest14443-21ls10:35:151
3619991810,6cyclictest3359-21sshd12:34:161
3619991810,6cyclictest28480-21sshd11:54:251
3619991810,6cyclictest0-21swapper/112:00:231
3619991810,5cyclictest25901-21kthreadcore10:40:211
3619991810,5cyclictest25901-21kthreadcore10:40:211
3619991810,5cyclictest13241-21kthreadcore11:45:211
3619991810,5cyclictest12722-21sh11:10:131
361999180,9cyclictest5324-21grep11:05:201
360899185,10cyclictest1-21systemd09:53:490
360899184,3cyclictest131rcu_preempt11:10:320
360899181,9cyclictest0-21swapper/011:30:220
360899181,3cyclictest0-21swapper/011:07:560
360899181,14cyclictest5198-21kthreadcore07:10:200
3608991811,4cyclictest21710-21diskmemload10:15:120
360899181,13cyclictest2537-21ssh09:35:140
3608991810,6cyclictest9393-21cron12:20:010
360899180,14cyclictest20424-21ls09:05:220
363799179,6cyclictest0-21swapper/310:30:113
363799172,9cyclictest0-21swapper/308:50:133
363799172,10cyclictest0-21swapper/308:30:153
3637991711,4cyclictest0-21swapper/309:45:113
363799171,12cyclictest321rcuc/309:00:133
361999179,6cyclictest17909-21cron07:25:011
361999179,6cyclictest1308-21cut10:10:161
361999179,5cyclictest24559-21grep12:10:171
361999178,7cyclictest21252-21cut08:35:181
361999177,7cyclictest14309-21cron07:55:011
361999173,12cyclictest10294-21sed12:20:171
361999173,11cyclictest8153-21kthreadcore10:30:211
361999171,6cyclictest4391-21/usr/sbin/munin07:10:181
361999171,6cyclictest16059-21apt-get11:30:021
361999171,13cyclictest7898-21df09:20:151
361999171,13cyclictest12379-21apt-get09:00:131
361999171,12cyclictest855-21df12:15:151
361999171,12cyclictest10413-21cut10:15:151
3619991710,5cyclictest21-21ksoftirqd/107:25:161
3619991710,5cyclictest21-21ksoftirqd/107:25:161
360899178,7cyclictest17881-21users07:20:300
360899176,3cyclictest12-21ksoftirqd/009:40:230
360899171,4cyclictest0-21swapper/010:07:350
3608991711,4cyclictest0-21swapper/007:25:290
3608991711,4cyclictest0-21swapper/007:25:290
360899171,13cyclictest27771-21ssh12:30:100
360899171,12cyclictest6151-21ssh12:00:120
360899171,12cyclictest0-21swapper/008:05:140
360899171,12cyclictest0-21swapper/008:05:140
3608991710,5cyclictest25841-21ssh09:10:280
3608991710,5cyclictest12109-21sshd08:57:250
363799162,9cyclictest0-21swapper/308:45:143
361999169,5cyclictest454-21kthreadcore09:50:231
361999163,10cyclictest3683-21kthreadcore08:50:191
361999162,9cyclictest0-21swapper/107:30:151
361999162,10cyclictest19125-21apt-get08:00:021
361999162,10cyclictest19125-21apt-get08:00:021
361999161,7cyclictest19548-21cut09:05:211
361999161,12cyclictest4954-21apt-get07:45:151
361999161,12cyclictest20249-21ssh10:20:191
361999161,12cyclictest1525-21apt-get08:15:141
361999161,12cyclictest1242-21apt-config08:15:011
361999161,12cyclictest10704-21kthreadcore07:15:221
361999161,11cyclictest14571-21ntp_192.168.11408:25:261
361999161,11cyclictest10226-21kthreadcore08:55:221
361999161,10cyclictest15549-21ssh10:00:131
361999161,10cyclictest15549-21ssh10:00:131
3619991610,4cyclictest27039-21sh11:00:121
3619991610,4cyclictest0-21swapper/108:30:251
360899168,6cyclictest32255-21kthreadcore08:45:220
360899168,6cyclictest31164-21sshd07:35:250
360899168,6cyclictest20849-21entropy08:35:160
360899168,6cyclictest16237-21if_err_eth108:30:190
3608991611,3cyclictest7001-21kernelversion08:20:200
3608991611,3cyclictest7001-21kernelversion08:20:200
3608991611,2cyclictest0-21swapper/008:00:110
360899161,11cyclictest13290-21grep07:20:020
363799158,5cyclictest35966irq/58-eth007:50:253
363799152,9cyclictest0-21swapper/308:10:153
361999151,7cyclictest31126-21ntp_kernel_err07:35:241
361999151,11cyclictest25584-21apt-get08:40:141
360899157,6cyclictest29377-21tail08:10:190
360899151,11cyclictest25710-21grep08:40:160
360899151,11cyclictest1991-21tail07:40:210
360899151,11cyclictest12114-21kthreadcore08:25:220
363799141,9cyclictest0-21swapper/308:20:203
363799141,9cyclictest0-21swapper/308:20:203
363799141,10cyclictest0-21swapper/307:35:163
361999141,11cyclictest3631-21latency_hist07:10:021
360899141,11cyclictest3087-21if_eth108:50:170
360899141,10cyclictest3596-21cyclictest07:10:020
363799132,9cyclictest0-21swapper/307:15:153
36379991,6cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional