You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-20 - 16:13

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot7s.osadl.org (updated Thu Aug 20, 2026 12:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1092328875,9sleep00-21swapper/007:09:440
1069328774,9sleep20-21swapper/207:06:442
1080827966,9sleep30-21swapper/307:08:143
874927658,13sleep10-21swapper/107:05:031
1104699290,28cyclictest0-21swapper/209:59:282
1104699290,28cyclictest0-21swapper/209:11:122
1104699250,24cyclictest0-21swapper/209:18:462
1104699230,22cyclictest0-21swapper/211:40:512
1104699230,21cyclictest0-21swapper/211:59:232
1104899227,10cyclictest24006-21apache_volume12:15:103
11048992210,8cyclictest225-21systemd-journal08:41:543
1104699222,8cyclictest0-21swapper/207:20:132
1104699220,22cyclictest0-21swapper/212:23:152
36582210,0sleep10-21swapper/110:26:061
1104699200,20cyclictest0-21swapper/207:29:302
1104699200,19cyclictest0-21swapper/211:35:462
1104299204,15cyclictest3995-21diskmemload11:14:241
1104299204,15cyclictest3995-21diskmemload11:14:241
11048991918,1cyclictest0-21swapper/310:53:253
1104899190,18cyclictest0-21swapper/309:54:543
1104699190,19cyclictest0-21swapper/210:33:162
1104699190,17cyclictest0-21swapper/210:46:292
1104299190,3cyclictest21119-21ssh12:32:111
1104899182,12cyclictest12857-21apt-get09:15:113
1104699185,13cyclictest0-21swapper/208:30:092
1104699176,11cyclictest0-21swapper/208:55:592
1104699170,17cyclictest0-21swapper/212:05:292
1104699170,16cyclictest0-21swapper/209:38:342
1104699170,15cyclictest0-21swapper/209:49:372
1104699170,15cyclictest0-21swapper/209:21:082
11042991716,1cyclictest0-21swapper/110:58:011
1104299171,0cyclictest14824-21ssh11:30:231
1104299170,1cyclictest16645-21if_enp0s31f612:10:181
1103999175,6cyclictest29070-21munin-run09:24:590
1104899160,15cyclictest0-21swapper/312:02:483
11046991614,2cyclictest0-21swapper/210:37:122
1104699160,16cyclictest0-21swapper/212:11:282
1104699160,16cyclictest0-21swapper/212:00:122
1104699160,16cyclictest0-21swapper/211:14:592
1104699160,16cyclictest0-21swapper/211:14:592
1104699160,14cyclictest0-21swapper/212:19:332
1104299162,9cyclictest225-21systemd-journal08:24:591
1104299160,15cyclictest0-21swapper/111:23:241
1104899150,11cyclictest0-21swapper/312:30:133
11046991514,1cyclictest0-21swapper/209:30:072
1104699150,1cyclictest0-21swapper/211:06:332
1104299159,2cyclictest0-21swapper/109:46:071
11042991513,2cyclictest0-21swapper/110:15:541
1103999153,7cyclictest17360-21cron08:30:000
1103999152,7cyclictest21700-21dpkg09:40:000
1103999150,13cyclictest30678-21cron07:49:590
1104899141,8cyclictest18861-21/usr/sbin/munin10:55:133
11048991410,3cyclictest13539-21grep11:30:143
11048991410,2cyclictest12615-21turbostat08:20:003
1104899140,1cyclictest0-21swapper/309:37:093
11046991413,1cyclictest0-21swapper/210:15:352
11046991411,2cyclictest23538-21cat07:34:592
11046991410,3cyclictest5606-21ssh10:08:412
1104699140,13cyclictest29994-21wc07:45:252
11042991414,0cyclictest0-21swapper/109:50:581
1104299141,12cyclictest24452-21ssh10:39:121
11042991410,3cyclictest21255-21cut07:30:121
1104299140,2cyclictest0-21swapper/111:28:131
1104299140,14cyclictest0-21swapper/110:02:121
1104299140,13cyclictest0-21swapper/109:21:051
1103999146,2cyclictest29575-21awk11:40:010
1103999141,7cyclictest14568-21munin-run09:55:000
11039991410,3cyclictest11163-21grep10:30:260
1104899139,3cyclictest4983-21sed08:00:273
11048991310,2cyclictest8059-21ssh12:05:183
11048991310,2cyclictest7918-21ssh11:45:263
11048991310,2cyclictest31658-21grep07:50:143
11048991310,2cyclictest2478-21sort07:55:263
11048991310,2cyclictest23443-21sh11:35:243
11048991310,2cyclictest20394-21rm11:15:143
11048991310,2cyclictest1881-21cat10:25:183
11048991310,2cyclictest15213-21grep11:50:173
1104899130,12cyclictest8294-21grep08:10:153
1104899130,12cyclictest23211-21head09:40:163
1104699139,3cyclictest29877-21tr09:25:142
1104699139,3cyclictest27119-21ls10:40:182
1104699133,10cyclictest29176-21cstates11:20:132
1104699132,9cyclictest12070-21/usr/sbin/munin10:50:252
11046991310,2cyclictest7920-21tr09:50:212
11046991310,2cyclictest32709-21ls10:25:002
11046991310,2cyclictest31756-21chrt11:02:262
11046991310,2cyclictest25886-21df12:35:172
11046991310,2cyclictest22954-21grep08:40:172
11046991310,2cyclictest16318-21grep11:50:262
11046991310,2cyclictest12908-21sed11:30:002
1104699130,0cyclictest0-21swapper/210:01:352
1104299139,3cyclictest17966-21grep07:20:251
1104299139,3cyclictest15055-21ls07:15:231
1104299139,3cyclictest10391-21grep12:25:261
1104299135,4cyclictest225-21systemd-journal11:45:011
1104299132,1cyclictest16872-21kworker/1:210:40:311
1104299131,2cyclictest29329-21sh11:00:321
11042991310,2cyclictest7402-21grep12:05:151
11042991310,2cyclictest25344-21cut12:35:151
11042991310,2cyclictest25193-21grep07:35:221
1104299130,12cyclictest0-21swapper/109:35:211
1103999139,3cyclictest18373-21grep08:30:180
1103999139,2cyclictest32147-21grep10:05:150
1103999138,3cyclictest19687-21head10:35:290
11039991312,1cyclictest0-21swapper/010:15:300
11039991312,0cyclictest0-21swapper/011:54:330
11039991310,2cyclictest24511-21sh08:45:000
11039991310,2cyclictest16932-21grep07:20:150
11039991310,2cyclictest10224-21dump-pmu-power08:15:000
1103999130,1cyclictest141rcu_preempt11:15:370
1103999130,12cyclictest25514-21sed07:35:260
1104899129,2cyclictest731-21grep12:39:593
1104899129,2cyclictest30937-21ls09:25:213
1104899129,2cyclictest25638-21cut10:20:173
1104899129,2cyclictest25493-21cron10:40:013
1104899129,2cyclictest22060-21head09:20:173
1104899129,2cyclictest17347-21grep07:20:203
1104899129,2cyclictest17259-21wc10:35:013
1104899129,2cyclictest13600-21head08:20:163
1104899128,2cyclictest31060-21ssh09:45:143
1104899124,6cyclictest2297-21sh11:23:263
11048991210,1cyclictest19109-21grep07:25:143
1104699129,2cyclictest7764-21wc11:45:252
1104699129,2cyclictest6388-21grep08:05:182
1104699129,2cyclictest2137-21grep09:05:132
1104699129,2cyclictest104802sleep208:15:122
1104699128,3cyclictest18911-21grep10:55:132
1104699128,3cyclictest16996-21aten2_r7power_v12:30:142
1104699127,3cyclictest24848-21grep08:45:122
1104699120,9cyclictest0-21swapper/210:10:092
1104299129,2cyclictest9536-21tr10:30:151
1104299129,2cyclictest5731-21tr09:30:141
1104299129,2cyclictest16322-21uname09:55:191
1104299129,2cyclictest11281-21cut10:50:171
1104299129,2cyclictest11216-21head08:15:171
1104299128,3cyclictest8303-21ssh11:45:411
1104299128,3cyclictest13604-21perf07:15:011
1104299126,5cyclictest8124-21ssh10:48:401
1104299123,5cyclictest225-21systemd-journal10:25:001
1104299122,6cyclictest19737-21cron08:35:001
1104299120,9cyclictest7730-21/usr/sbin/munin10:10:211
1104299120,9cyclictest32600-21grep07:50:231
1104299120,12cyclictest0-21swapper/112:04:101
1104299120,12cyclictest0-21swapper/111:36:021
1104299120,11cyclictest31282-21wc09:25:251
1104299120,11cyclictest22478-21sh11:54:591
1104299120,11cyclictest0-21swapper/109:10:191
1103999129,2cyclictest8439-21ssh12:25:130
1103999129,2cyclictest8152-21sed12:05:170
1103999129,2cyclictest3844-21df08:00:160
1103999129,2cyclictest32739-21wc10:25:010
1103999129,2cyclictest2636-21sed09:05:170
1103999129,2cyclictest18303-21wc12:30:210
1103999129,2cyclictest1642-21ssh10:25:170
1103999129,2cyclictest15735-21turbostat10:15:000
1103999129,2cyclictest12595-21cron08:20:000
1103999129,2cyclictest12203-21wc11:10:140
1103999129,2cyclictest12203-21wc11:10:140
1103999128,3cyclictest6654-21ls09:30:200
1103999128,3cyclictest22833-21grep07:30:250
1103999126,5cyclictest4346-21perf11:25:010
1103999123,5cyclictest3995-21diskmemload12:02:580
1103999120,2cyclictest0-21swapper/009:46:260
1104899119,1cyclictest27453-21wc10:40:213
1104899119,1cyclictest14542-21ssh10:14:053
1104899118,2cyclictest9175-21ssh09:12:323
1104899118,2cyclictest6353-21sort08:05:173
1104899118,2cyclictest6117-21grep11:25:223
1104899118,2cyclictest609-21tr09:00:203
1104899118,2cyclictest3551-21taskset10:07:063
1104899118,2cyclictest3498-21tr11:05:143
1104899118,2cyclictest32267-21ssh11:02:493
1104899118,2cyclictest3214-21tail10:45:203
1104899118,2cyclictest32031-21grep11:40:273
1104899118,2cyclictest30298-21tail08:55:183
1104899118,2cyclictest30148-21wc07:45:253
1104899118,2cyclictest26758-21chrt08:48:313
1104899118,2cyclictest26735-21ssh10:01:263
1104899118,2cyclictest26605-21ssh11:56:433
1104899118,2cyclictest21219-21apache_processe07:30:123
1104899118,2cyclictest17559-21wc08:30:013
1104899118,2cyclictest15784-21cut12:10:133
1104899118,2cyclictest15688-21grep09:55:163
1104899118,2cyclictest14531-21ssh11:10:583
1104899118,2cyclictest14531-21ssh11:10:573
1104899118,2cyclictest14224-21ssh12:28:203
1104899117,3cyclictest2588-21tail09:05:163
1104899115,2cyclictest225-21systemd-journal08:55:013
1104899112,7cyclictest16191-21/usr/sbin/munin10:15:113
1104899110,10cyclictest6071-21sed09:30:163
1104899110,10cyclictest3871-21ssh12:21:523
1104899110,10cyclictest26031-21sed07:40:003
1104699118,2cyclictest9046-21ssh12:25:152
1104699118,2cyclictest32205-21wc07:50:192
1104699118,2cyclictest27851-21sed07:40:262
1104699118,2cyclictest27754-21sort08:50:182
1104699118,2cyclictest25227-21cat07:35:252
1104699118,2cyclictest24951-21ssh09:40:512
1104699118,2cyclictest20186-21cat11:15:112
1104699118,2cyclictest1837-21ssh10:25:182
1104699118,2cyclictest15945-21grep08:25:152
1104699118,2cyclictest14102-21tail11:30:162
1104699117,3cyclictest4495-21taskset08:00:222
1104299118,2cyclictest6080-21expr08:05:171
1104299118,2cyclictest5438-21wc08:05:001
1104299118,2cyclictest4736-21tr11:05:221
1104299118,2cyclictest3005-21cron07:59:591
1104299118,2cyclictest28138-21awk11:20:001
1104299118,2cyclictest26112-21ssh09:41:461
1104299118,2cyclictest23806-21ssh11:55:151
1104299118,2cyclictest20662-21seq07:26:261
1104299118,2cyclictest15284-21cut08:25:121
1104299118,2cyclictest1414-21sed10:05:271
1104299117,3cyclictest23627-21head08:40:241
1104299110,10cyclictest94-21kswapd012:20:121
1104299110,10cyclictest21405-21python08:35:261
1103999119,1cyclictest31545-21sort07:50:130
1103999118,2cyclictest9015-21ssh11:46:140
1103999118,2cyclictest4496-21ssh12:22:210
1103999118,2cyclictest32516-21grep09:00:160
1103999118,2cyclictest31207-21ssh11:40:180
1103999118,2cyclictest310-21ssh11:03:130
1103999118,2cyclictest2980-21ssh11:05:040
1103999118,2cyclictest29543-21apache_accesses09:25:110
1103999118,2cyclictest29344-21ssh10:41:270
1103999118,2cyclictest27529-21ssh09:42:540
1103999118,2cyclictest26954-21sed07:40:170
1103999118,2cyclictest25742-21grep12:15:220
1103999118,2cyclictest25606-21grep12:35:160
1103999118,2cyclictest24490-21sendmail_mailtr11:55:210
1103999118,2cyclictest24083-21ssh10:00:160
1103999118,2cyclictest20471-21ssh12:12:350
1103999118,2cyclictest19872-21ssh09:19:200
1103999118,2cyclictest19380-21df10:55:130
1103999118,2cyclictest1868-21ssh10:45:090
1103999118,2cyclictest15617-21ssh09:55:150
1103999118,2cyclictest13099-21ssh11:30:070
1103999118,2cyclictest12952-21ssh10:50:510
1103999118,2cyclictest10908-21ssh11:28:350
1103999117,3cyclictest3219-21wc07:59:590
1104899108,1cyclictest218012sleep308:36:053
1104899108,1cyclictest14793-21cut07:15:203
1104899108,1cyclictest133332sleep307:11:543
1104899107,2cyclictest26257-21cut07:40:123
1104899107,2cyclictest17774-21grep08:30:143
1104699108,1cyclictest217582sleep208:35:312
1104699108,1cyclictest14929-21cat07:15:212
1104699107,2cyclictest9412-21systemctl08:10:232
1104699107,2cyclictest1184-21grep09:00:262
1104699101,8cyclictest889-21apache_processe07:55:122
1104699100,9cyclictest0-21swapper/208:20:102
1104699100,9cyclictest0-21swapper/207:10:102
1104299108,1cyclictest31859-21cat09:00:001
1104299108,1cyclictest27152-21apache_processe08:50:141
1104299108,1cyclictest25202-21ls08:45:131
1104299108,1cyclictest13236-21expr09:15:141
1104299107,2cyclictest8672-21cut08:10:171
1104299107,2cyclictest750-21cut09:00:221
1104299107,2cyclictest29393-21sort07:45:171
1104299107,2cyclictest27466-21wc07:40:211
1104299102,4cyclictest225-21systemd-journal09:05:341
1104299100,4cyclictest25414-21tr12:15:181
1103999108,1cyclictest5667-21basename08:05:130
1103999108,1cyclictest5453-21if_enp0s31f609:10:180
1103999108,1cyclictest27779-21sort08:50:180
1103999107,2cyclictest30150-21grep08:55:170
1103999107,2cyclictest20385-21cat08:35:160
1103999107,2cyclictest11844-21expr07:10:120
1103999106,3cyclictest25401-21grep08:45:140
1103999106,3cyclictest19110-21tr07:25:140
110399997,1cyclictest14596-21head07:15:180
110399997,1cyclictest13534-21sort08:20:150
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional