You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-20 - 19:33

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot4s.osadl.org (updated Mon Apr 20, 2026 12:45:09)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2875099613,33cyclictest21863-21ssh09:50:132
2875899494,26cyclictest35966irq/58-eth009:10:143
2875099480,34cyclictest19739-21ssh10:25:142
2875099480,34cyclictest19739-21ssh10:25:132
2873399462,37cyclictest28655-21ssh10:30:110
2875099451,31cyclictest23874-21ssh09:15:152
2875099443,32cyclictest28670-21ssh10:30:122
2875899431,26cyclictest35966irq/58-eth009:30:133
2875899412,26cyclictest35966irq/58-eth008:10:133
2875899411,27cyclictest35966irq/58-eth010:45:123
28758994112,6cyclictest35966irq/58-eth010:10:143
28739994033,5cyclictest0-21swapper/111:55:141
2873999402,27cyclictest18440-21ssh09:30:131
2875899392,23cyclictest35966irq/58-eth011:45:143
2875099392,20cyclictest17447-21/usr/sbin/munin11:00:142
2875099381,22cyclictest16741-21ssh12:30:152
2873999374,19cyclictest15410-21ssh11:35:131
2873999362,31cyclictest25707-21ssh12:35:161
2873999360,3cyclictest131rcu_preempt07:35:121
2873399361,27cyclictest9451-21ssh09:25:140
2873399361,27cyclictest9451-21ssh09:25:130
2875099352,27cyclictest10163-21ssh11:13:142
2875099351,28cyclictest14845-21ssh10:57:132
2873999357,16cyclictest27789-21ssh12:00:131
2873999353,29cyclictest9905-21sshd09:05:141
28733993514,11cyclictest28721-21cyclictest07:30:120
2873999343,26cyclictest18196-21apt-get08:40:131
2873399342,25cyclictest31288-21ssh12:20:130
2875899333,14cyclictest35966irq/58-eth011:30:113
28739993321,9cyclictest21-21ksoftirqd/109:40:131
28739993320,10cyclictest32530-1kworker/u9:111:45:131
2875099324,20cyclictest27-21ksoftirqd/211:55:142
2873999323,24cyclictest21033-21ssh11:20:141
28739993212,10cyclictest0-21swapper/109:20:131
2873399324,19cyclictest14856-21ssh09:10:140
2875899315,16cyclictest35966irq/58-eth009:50:143
2875099311,18cyclictest4333-21ssh12:05:132
2873999314,22cyclictest29963-21sh11:25:131
2873999311,26cyclictest23151-21ssh10:45:121
2873999311,23cyclictest10878-21sshd07:25:141
28758993014,11cyclictest35966irq/58-eth011:00:133
2873999303,20cyclictest22290-21ssh12:15:151
2873399303,21cyclictest19728-21ssh10:25:140
2873399303,21cyclictest19728-21ssh10:25:130
2873399302,21cyclictest32086-21ssh10:50:140
2875899293,13cyclictest0-21swapper/311:25:263
2875099292,21cyclictest20398-21sh12:13:132
2873999293,16cyclictest21-21ksoftirqd/108:25:121
2873999291,22cyclictest8569-21ssh10:55:131
2873999291,18cyclictest0-21swapper/111:10:131
2875899283,18cyclictest0-21swapper/310:56:143
28758992810,6cyclictest35966irq/58-eth010:00:123
2875099282,19cyclictest27458-21ssh09:35:162
2873999284,19cyclictest201rcuc/112:10:131
2873999282,19cyclictest12029-21ssh11:15:131
2875899277,16cyclictest35966irq/58-eth010:30:113
2875899272,11cyclictest0-21swapper/312:15:483
2875099275,15cyclictest27346irq/40-dwc2_hso08:20:112
2875099273,17cyclictest0-21swapper/212:20:132
2875099271,20cyclictest10643-21ssh10:20:142
28750992711,13cyclictest0-21swapper/208:45:152
2873999272,22cyclictest22422-21ssh11:00:331
2873999272,19cyclictest7837-21sh12:25:141
28739992715,7cyclictest21-21ksoftirqd/111:40:141
2873999271,21cyclictest32077-21ssh10:50:121
28733992720,3cyclictest12-21ksoftirqd/012:25:120
2873399271,19cyclictest16731-21sshd12:30:140
2875899265,10cyclictest0-21swapper/311:05:143
2875899262,8cyclictest0-21swapper/309:45:163
2875899262,10cyclictest0-21swapper/311:12:143
28758992616,6cyclictest35966irq/58-eth012:25:143
2875099263,15cyclictest27346irq/40-dwc2_hso07:30:132
2875099262,18cyclictest0-21swapper/207:55:142
28750992612,6cyclictest21028-21sshd11:20:142
28750992611,12cyclictest0-21swapper/208:25:262
2873999263,19cyclictest201rcuc/111:05:131
2873999263,16cyclictest0-21swapper/109:15:151
2873399263,19cyclictest18849-21ssh11:55:150
2873399261,4cyclictest131rcu_preempt11:35:130
2875899253,15cyclictest35966irq/58-eth010:25:133
2875899253,15cyclictest35966irq/58-eth010:25:133
2875899253,10cyclictest0-21swapper/312:11:143
2875899253,10cyclictest0-21swapper/312:05:143
28758992514,8cyclictest0-21swapper/309:55:173
2875099252,8cyclictest7807-21apt-get12:25:142
28750992513,9cyclictest15361-21ssh09:26:472
28750992513,9cyclictest15361-21ssh09:26:472
28750992513,5cyclictest0-21swapper/208:40:232
2875099251,18cyclictest22256-21apt-get12:15:142
28750992511,10cyclictest15031-21cron11:35:022
28750992510,12cyclictest27111-21irqrtprio11:05:192
2873999259,12cyclictest0-21swapper/110:40:121
2873999252,18cyclictest30834-21ssh09:55:131
2873999251,19cyclictest1550-21ssh10:15:121
28739992510,12cyclictest28721-21cyclictest11:50:141
2873399252,18cyclictest26550-21sshd11:05:140
2875899241,17cyclictest0-21swapper/310:54:133
2875099242,16cyclictest0-21swapper/211:15:142
2873999241,17cyclictest25053-21sshd10:10:141
2873399242,19cyclictest3095-21ssh11:10:130
28758992316,3cyclictest0-21swapper/309:15:153
2875099236,3cyclictest131rcu_preempt10:10:132
2875099232,16cyclictest0-21swapper/210:54:142
2875099231,9cyclictest1855-21idleruntime-cro12:40:002
2875099231,6cyclictest0-21swapper/210:05:152
2875099231,6cyclictest0-21swapper/210:05:152
2875099231,6cyclictest0-21swapper/207:50:132
28750992315,5cyclictest0-21swapper/207:15:222
28750992311,4cyclictest7340-21ssh10:00:122
2875099231,14cyclictest30332-21grep11:25:172
2875099231,13cyclictest28510-21irqstats12:00:192
28750992310,9cyclictest1368-21irqcore11:45:172
2873399231,16cyclictest4160-21ls09:40:170
2875899229,10cyclictest0-21swapper/310:15:143
2875099226,6cyclictest27-21ksoftirqd/209:55:152
2875099226,11cyclictest27346irq/40-dwc2_hso09:10:152
2875099223,12cyclictest27346irq/40-dwc2_hso07:10:132
2875099222,11cyclictest11940-1kworker/u9:010:45:122
28750992214,5cyclictest0-21swapper/211:40:252
28750992211,8cyclictest0-21swapper/207:25:162
2875099221,14cyclictest14442-21diskmemload09:33:522
2875099221,10cyclictest3391-21kthreadcore09:20:232
2873999222,14cyclictest201rcuc/108:35:131
2873999221,2cyclictest131rcu_preempt10:35:161
28739992212,7cyclictest28195-21kthreadcore09:35:211
2873399228,3cyclictest131rcu_preempt11:00:110
2873399226,13cyclictest987-21ssh11:45:140
2873399222,3cyclictest131rcu_preempt10:55:120
2873399222,2cyclictest131rcu_preempt09:20:140
2873399222,17cyclictest12884-21ssh09:45:140
2873399221,18cyclictest28876-21apt-get07:10:110
2873399220,19cyclictest13319-21ssh12:10:140
2875099219,9cyclictest11960-21ssh09:44:242
2875099212,9cyclictest6079-21kthreadcore09:00:182
2875099211,9cyclictest29896-21wc08:15:232
2875099211,8cyclictest21991-21fschecks_time08:10:162
28750992114,4cyclictest0-21swapper/210:35:022
28750992112,6cyclictest23977-21cron11:40:012
2875099211,16cyclictest6254-21fschecks_time07:20:162
2875099211,16cyclictest16389-21wc09:45:252
2875099211,11cyclictest5610-21timerwakeupswit10:15:292
2875099211,10cyclictest21459-21kthreadcore07:35:202
2873999212,16cyclictest22364-21ssh10:25:231
2873999212,16cyclictest22364-21ssh10:25:221
28739992114,5cyclictest0-21swapper/109:45:171
28739992113,4cyclictest4587-21hwlatdetect12:05:151
2873999211,17cyclictest16911-21ls08:05:021
2873999211,16cyclictest7338-21sh10:00:131
2873999211,16cyclictest73-21kswapd011:30:121
2873399212,14cyclictest4433-21ssh12:05:140
2873399211,5cyclictest0-21swapper/010:42:140
2873399211,16cyclictest23130-21find10:45:110
2875899203,11cyclictest0-21swapper/312:20:133
2875899201,13cyclictest0-21swapper/308:00:293
2875099202,12cyclictest27346irq/40-dwc2_hso07:40:132
2875099201,9cyclictest20882-21grep08:05:302
2875099201,9cyclictest11329-21kthreadcore09:05:202
2875099201,9cyclictest10093-21cut11:50:162
2875099201,15cyclictest30819-21kthreadcore07:45:192
2875099201,10cyclictest3251-21kthreadcore08:55:222
2875099200,10cyclictest16990-21sort08:05:022
28739992012,5cyclictest17788-21kthreadcore09:10:221
2873999201,12cyclictest18049-21latency_hist09:30:011
2873999201,12cyclictest18049-21latency_hist09:30:001
2873999200,17cyclictest4907-21ls09:00:021
28733992012,5cyclictest22582-21grep09:30:300
2873399201,15cyclictest24416-21ssh11:40:150
28733992011,5cyclictest21879-21sh09:50:150
2873399201,14cyclictest21480-21missed_timers08:40:240
2873399200,18cyclictest19433-21ls10:25:010
2873399200,14cyclictest27618-21uniq09:35:180
2875899198,8cyclictest0-21swapper/311:56:453
2875899198,8cyclictest0-21swapper/311:53:133
2875899193,11cyclictest0-21swapper/311:20:133
2875899193,10cyclictest35966irq/58-eth011:35:143
28758991911,6cyclictest0-21swapper/311:40:203
2875899191,12cyclictest0-21swapper/308:35:153
2875899191,12cyclictest0-21swapper/307:55:163
2875899191,11cyclictest0-21swapper/308:55:143
28758991910,6cyclictest35966irq/58-eth011:15:143
28758991910,6cyclictest0-21swapper/309:40:353
2875099191,9cyclictest9109-21grep08:30:172
2875099191,9cyclictest17938-21cron08:40:012
2875099191,9cyclictest14224-21/usr/sbin/munin10:40:192
2873999199,8cyclictest6097-21apt-get07:20:121
2873999199,7cyclictest28780-21df_inode10:30:121
2873999198,8cyclictest32237-21cat12:20:171
2873999195,9cyclictest21-21ksoftirqd/108:30:131
2873999191,9cyclictest27930-21fschecks_time08:50:161
2873999191,5cyclictest53646usb-storage10:20:111
2873999191,3cyclictest0-21swapper/107:45:131
2873999191,15cyclictest29896-21cron07:45:011
2873999191,13cyclictest16853-21diskstats12:30:151
2873999191,12cyclictest21958-21sh09:50:151
2873399197,3cyclictest131rcu_preempt11:35:010
28733991914,3cyclictest0-21swapper/007:35:140
28733991913,4cyclictest25055-21ssh10:10:140
28733991912,5cyclictest1-21systemd09:05:160
2873399191,14cyclictest2020-21ntp_192.168.11409:55:250
28733991910,6cyclictest13849-21cron10:40:010
2873399190,2cyclictest21007-21ssh11:20:120
2873399190,16cyclictest5050-21ls09:00:120
2875899189,6cyclictest0-21swapper/310:08:143
2875899189,6cyclictest0-21swapper/310:08:133
2875899189,6cyclictest0-21swapper/309:39:403
2875899189,6cyclictest0-21swapper/309:20:123
2875899188,8cyclictest0-21swapper/310:38:313
2875899182,11cyclictest0-21swapper/312:30:143
2875099181,8cyclictest27896-21fschecks_count08:50:152
2873999188,6cyclictest0-21swapper/108:15:131
28739991811,5cyclictest0-21swapper/108:20:171
2873999181,13cyclictest17431-21cut10:05:211
2873999181,13cyclictest17431-21cut10:05:201
2873399184,9cyclictest24130-21ls09:15:170
2873399184,12cyclictest151rcuc/012:00:120
2873399182,14cyclictest18170-1kworker/u9:111:25:120
2873399181,9cyclictest9466-21cut07:55:210
2873399181,8cyclictest5876-21sendmail-msp07:20:010
2873399181,13cyclictest1527-21apt10:15:120
28733991810,6cyclictest22280-21sshd12:15:140
28733991810,5cyclictest16914-21kthreadcore10:05:220
28733991810,5cyclictest16914-21kthreadcore10:05:210
2875899179,6cyclictest0-21swapper/310:40:073
2875899177,3cyclictest0-21swapper/312:00:273
2875899172,12cyclictest35966irq/58-eth010:20:143
2875899172,10cyclictest35966irq/58-eth007:15:123
2875899172,10cyclictest0-21swapper/309:25:143
2875899172,10cyclictest0-21swapper/309:25:143
2875899172,10cyclictest0-21swapper/308:30:143
2875899172,10cyclictest0-21swapper/308:20:133
2875899172,10cyclictest0-21swapper/307:30:153
2875899172,10cyclictest0-21swapper/307:20:123
28758991710,5cyclictest35966irq/58-eth012:35:153
2875099171,13cyclictest28759-21latency_hist07:10:022
2873999174,11cyclictest24484-21tr08:10:211
2873999174,10cyclictest16803-21kthreadcore07:30:201
2873999173,6cyclictest29828-21kthreadcore07:10:191
2873999172,13cyclictest17533-21irqrtprio08:05:181
28739991711,4cyclictest0-21swapper/107:10:021
2873999171,13cyclictest6062-21kthreadcore09:00:181
2873999171,13cyclictest2000-21cut07:15:171
2873399179,6cyclictest32318-21sshd08:54:140
2873399179,6cyclictest25543-21wc07:40:170
2873399178,7cyclictest5946-21ntp_192.168.11407:50:230
2873399177,7cyclictest14561-21kthreadcore08:35:210
2873399172,11cyclictest12013-21apt-get11:15:140
2873399171,14cyclictest29560-21ssh12:35:300
28733991711,4cyclictest14752-21unixbench_singl07:25:290
2873399171,14cyclictest13159-21ls08:35:010
28733991710,5cyclictest8366-21sshd08:28:150
28733991710,5cyclictest26956-21irqcore08:15:190
28733991710,4cyclictest0-21swapper/008:20:170
2873399170,3cyclictest0-21swapper/011:50:100
2875899162,7cyclictest0-21swapper/307:25:123
2875899162,11cyclictest0-21swapper/307:50:143
2875899162,10cyclictest0-21swapper/308:50:143
2875899162,10cyclictest0-21swapper/308:25:133
2875899162,10cyclictest0-21swapper/307:10:133
2875899161,10cyclictest0-21swapper/308:40:153
2873999163,11cyclictest2675-21grep07:50:161
2873999161,12cyclictest1-21systemd08:45:031
28739991610,4cyclictest7658-21if_eth107:55:171
2873399168,6cyclictest2268-21sshd07:46:140
2873399165,8cyclictest7654-21if_err_eth110:00:160
2873399161,13cyclictest22020-21fschecks_time08:10:150
2873399161,12cyclictest12209-21sshd08:00:130
2873399161,11cyclictest7298-21kthreadcore07:20:180
28733991610,4cyclictest17711-21kthreadcore08:05:200
2875899152,10cyclictest0-21swapper/308:15:143
2875899151,9cyclictest0-21swapper/307:45:143
2875899151,12cyclictest0-21swapper/308:05:173
2875899151,10cyclictest0-21swapper/309:05:143
2873399151,11cyclictest32658-21diskstats08:55:160
2873399151,11cyclictest24555-21kthreadcore08:45:200
2875899149,3cyclictest0-21swapper/308:45:283
2875899141,2cyclictest0-21swapper/307:40:143
2873399142,9cyclictest28732-21latency_hist07:10:020
2875899138,3cyclictest0-21swapper/309:00:263
2875899121,9cyclictest0-21swapper/307:35:223
2875899101,6cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional