You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-04 - 22:06

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Tue Aug 04, 2026 12:43:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2681729685,6sleep00-21swapper/007:06:390
2702729065,9sleep20-21swapper/207:09:222
2703628671,11sleep10-21swapper/107:09:291
2693928471,9sleep30-21swapper/307:08:143
27163992524,1cyclictest0-21swapper/012:29:540
2716399250,0cyclictest0-21swapper/009:35:450
2716999240,1cyclictest0-21swapper/209:34:372
27167992322,1cyclictest0-21swapper/109:16:041
2716799230,1cyclictest27312-21ntpq08:15:211
2717299220,1cyclictest0-21swapper/312:35:023
2716999220,1cyclictest0-21swapper/210:45:432
27169992119,1cyclictest22515-21ssh12:05:142
2716999210,20cyclictest0-21swapper/211:00:072
2716799210,21cyclictest0-21swapper/109:25:581
2717299202,11cyclictest0-21swapper/311:40:153
2717299202,11cyclictest0-21swapper/311:40:143
2716999209,3cyclictest477-21rs:main2
27169992019,1cyclictest0-21swapper/209:57:232
27169992019,1cyclictest0-21swapper/209:54:362
2716999200,20cyclictest0-21swapper/212:14:232
2716999200,19cyclictest0-21swapper/209:12:542
2716799200,1cyclictest0-21swapper/112:29:331
2716799200,1cyclictest0-21swapper/111:16:391
2716799200,19cyclictest0-21swapper/109:22:181
2716399208,6cyclictest12820-21munin-run08:55:010
2716399200,1cyclictest0-21swapper/010:33:010
2716999190,19cyclictest0-21swapper/211:53:432
2716799190,14cyclictest0-21swapper/109:14:301
27163991917,1cyclictest0-21swapper/011:26:140
2716399190,1cyclictest0-21swapper/010:00:080
2717299180,17cyclictest0-21swapper/309:28:443
27169991810,5cyclictest0-21swapper/211:45:522
2716999180,3cyclictest0-21swapper/212:35:002
2716999180,18cyclictest0-21swapper/211:31:482
2716999180,18cyclictest0-21swapper/209:25:592
2716999180,15cyclictest0-21swapper/211:23:432
2716799180,1cyclictest13307-21aten2_r7power_v11:40:161
2716799180,1cyclictest13307-21aten2_r7power_v11:40:161
2716799180,17cyclictest0-21swapper/109:48:411
27172991716,1cyclictest0-21swapper/311:06:313
2717299170,16cyclictest0-21swapper/311:15:213
27169991716,1cyclictest0-21swapper/212:29:552
2716999170,1cyclictest0-21swapper/211:07:482
27167991713,1cyclictest0-21swapper/112:12:551
2716399173,13cyclictest0-21swapper/008:35:130
27163991716,1cyclictest20178-21diskmemload12:38:050
2716399170,16cyclictest0-21swapper/010:49:360
2717299163,10cyclictest11511-21python10:40:263
2717299160,16cyclictest0-21swapper/311:14:113
27169991615,1cyclictest0-21swapper/210:41:422
2716999161,15cyclictest0-21swapper/210:08:022
2716999160,16cyclictest0-21swapper/209:49:222
2716999160,15cyclictest0-21swapper/209:35:462
2716799161,5cyclictest537-21snmpd10:50:051
2716799160,16cyclictest0-21swapper/110:36:581
2716399160,1cyclictest0-21swapper/009:26:430
2717299150,15cyclictest0-21swapper/308:17:153
27169991514,1cyclictest0-21swapper/212:32:462
2716999150,15cyclictest0-21swapper/209:41:252
2716799159,2cyclictest19469-21cut11:05:161
2716799155,10cyclictest20178-21diskmemload10:33:041
2716799152,10cyclictest20324-21/usr/sbin/munin11:25:221
27167991511,2cyclictest0-21swapper/111:11:381
2716799150,15cyclictest0-21swapper/112:17:531
2716799150,14cyclictest20178-21diskmemload10:17:081
2716799150,12cyclictest29767-21ls09:35:001
2716399157,7cyclictest31050-21munin-run09:55:000
2716399156,8cyclictest15410-21munin-run09:00:000
2716399150,1cyclictest0-21swapper/009:12:510
280742142,2sleep327157-21cyclictest11:49:043
2717299140,13cyclictest31877-21grep12:10:173
2717299140,12cyclictest0-21swapper/309:42:563
2716999144,7cyclictest14132-21apt-get12:00:132
2716999142,12cyclictest0-21swapper/209:24:552
27169991413,1cyclictest0-21swapper/212:23:532
27169991413,1cyclictest0-21swapper/210:57:592
27169991412,1cyclictest7619-21aten2_r7power_c10:00:162
27169991411,3cyclictest0-21swapper/211:18:502
2716999140,13cyclictest0-21swapper/210:30:402
2716799143,9cyclictest516-21gdbus07:40:001
2716799142,1cyclictest0-21swapper/110:23:041
2716799142,11cyclictest20178-21diskmemload11:00:181
27167991411,2cyclictest29315-21awk11:50:001
2716399149,5cyclictest8215-21apache212:11:070
2716399147,6cyclictest19851-21munin-run11:25:000
27163991413,1cyclictest0-21swapper/011:50:020
2716399140,10cyclictest23807-21/usr/sbin/munin12:05:220
2716399140,0cyclictest0-21swapper/010:44:480
169952140,0sleep30-21swapper/310:05:163
2717299139,3cyclictest20442-21grep11:25:153
2717299139,2cyclictest30154-21ssh11:30:213
2717299134,7cyclictest8248-21cups-browsed10:20:233
2717299132,7cyclictest8790-21runrttasks10:58:473
27172991310,2cyclictest30729-21ls08:25:003
27172991310,2cyclictest30524-21sed07:15:183
27172991310,2cyclictest28509-21cron09:15:003
27172991310,2cyclictest18007-21grep10:25:163
27172991310,2cyclictest15273-21wc07:50:183
27169991312,1cyclictest0-21swapper/210:38:232
27169991312,1cyclictest0-21swapper/210:24:232
27169991310,2cyclictest20746-21grep08:00:282
27169991310,2cyclictest17191-21cat09:00:272
2716999130,1cyclictest0-21swapper/211:28:592
2716999130,0cyclictest0-21swapper/212:15:492
2716799139,3cyclictest2374-21tr07:25:151
2716799139,3cyclictest16101-21sed12:00:281
2716799138,4cyclictest16791-21ssh10:44:181
2716799138,4cyclictest13237-21ssh12:37:571
2716799131,1cyclictest20178-21diskmemload10:25:031
27167991311,1cyclictest0-21swapper/108:45:121
27167991310,2cyclictest9623-21head10:00:291
27167991310,2cyclictest24582-21cut10:10:151
27167991310,2cyclictest19547-21sort09:05:271
2716799130,13cyclictest20178-21diskmemload10:09:021
2716799130,12cyclictest23368-21wc12:25:001
2716399139,3cyclictest20178-21diskmemload11:01:010
2716399135,5cyclictest0-21swapper/010:58:540
27163991310,2cyclictest20958-21cron11:45:000
27163991310,2cyclictest20958-21cron11:45:000
2717299129,2cyclictest5233-21head07:30:173
2717299129,2cyclictest470-21sed07:20:173
2717299129,2cyclictest32269-21sort09:55:183
2717299129,2cyclictest26337-21ls10:10:273
2717299129,2cyclictest23593-21cat12:05:203
2717299129,2cyclictest22196-21grep09:30:133
2717299129,2cyclictest21705-21cut08:05:143
2717299129,2cyclictest16105-21ssh12:20:163
2717299129,2cyclictest15387-21cut09:45:163
2717299129,2cyclictest13939-21sort08:55:183
2717299129,2cyclictest12318-21cut11:20:173
2717299128,3cyclictest18367-21sed07:55:293
2717299125,6cyclictest8279-21ssh10:00:193
2717299120,3cyclictest0-21swapper/310:30:203
2717299120,1cyclictest0-21swapper/312:30:103
2716999129,2cyclictest8373-21cat10:20:012
2716999129,2cyclictest7795-21ls08:45:002
2716999129,2cyclictest4001-21grep08:35:172
2716999129,2cyclictest28614-21grep07:10:212
2716999129,2cyclictest23946-21grep08:10:162
2716999129,2cyclictest22194-21grep08:05:172
2716999129,2cyclictest19104-21grep10:25:272
2716999129,2cyclictest14899-21sed08:55:272
2716999127,4cyclictest27552-21apache_accesses11:10:102
2716999124,7cyclictest13617-21munin-run12:00:002
27169991211,1cyclictest0-21swapper/210:50:222
27169991210,1cyclictest1661-21tr08:30:182
27169991210,1cyclictest1661-21tr08:30:172
2716999120,1cyclictest0-21swapper/207:25:152
2716999120,11cyclictest29816-21grep08:20:222
2716799129,2cyclictest4535-21cut07:30:141
2716799129,2cyclictest31382-21ssh09:35:201
2716799129,2cyclictest16732-21grep07:55:151
2716799129,2cyclictest15500-21grep07:50:201
2716799129,2cyclictest13872-21grep11:20:271
2716799128,3cyclictest849-21wc09:55:271
2716799128,3cyclictest31034-21sh09:55:001
2716799128,3cyclictest2003-21ssh11:52:301
2716799128,3cyclictest11885-21/usr/sbin/munin07:45:231
2716799128,2cyclictest1859-21sh07:25:001
2716799120,12cyclictest38150irq/131-snd_hda09:41:011
2716399129,2cyclictest25251-21grep10:10:180
2716399129,2cyclictest2264-21cut08:30:230
2716399129,2cyclictest2264-21cut08:30:230
2716399129,2cyclictest22616-21wc11:45:190
2716399128,3cyclictest759-21ssh10:54:050
2716399128,2cyclictest18434-21sh12:02:150
2716399124,7cyclictest14128-21perf09:45:000
2716399124,4cyclictest225-21systemd-journal12:25:000
2716399122,8cyclictest8826-21ssh10:20:120
27163991210,2cyclictest17326-21head10:05:190
2716399120,5cyclictest0-21swapper/012:18:570
2716399120,11cyclictest29010-21tr11:30:150
2716399120,11cyclictest27090-21cron11:10:000
2717299119,1cyclictest0-21swapper/310:52:223
2717299118,3cyclictest19957-21ssh12:03:263
2717299118,2cyclictest7655-21tune2fs12:15:153
2717299118,2cyclictest7445-21ssh09:20:393
2717299118,2cyclictest7222-21cat08:40:263
2717299118,2cyclictest718-21cut10:15:163
2717299118,2cyclictest7126-21tr07:35:143
2717299118,2cyclictest7053-21ntpq11:55:233
2717299118,2cyclictest6521-21timerwakeupswit11:35:293
2717299118,2cyclictest5639-21sh09:39:593
2717299118,2cyclictest3553-21grep08:35:153
2717299118,2cyclictest28905-21rm09:15:073
2717299118,2cyclictest25351-21ssh12:25:243
2717299118,2cyclictest23009-21ssh09:50:053
2717299118,2cyclictest1689-21head08:30:183
2717299118,2cyclictest1689-21head08:30:183
2717299118,2cyclictest16136-21expr09:00:163
2717299118,2cyclictest14228-21sh11:01:293
2717299118,2cyclictest12009-21gzip08:50:223
2717299118,2cyclictest11931-21hostname07:45:163
2717299118,2cyclictest10218-21sh08:50:013
2717299110,3cyclictest0-21swapper/310:45:133
2717299110,10cyclictest29881-21ssh11:50:143
2716999119,1cyclictest7632-21head07:35:172
2716999119,1cyclictest12416-21grep08:50:272
2716999118,3cyclictest5972-21ntp_kernel_err11:35:232
2716999118,2cyclictest9452-21tr08:45:212
2716999118,2cyclictest26415-21grep10:10:272
2716999118,2cyclictest15123-21if_enp0s31f607:50:162
2716999118,2cyclictest1470-21sed07:20:272
2716999118,2cyclictest10099-21head07:40:192
2716999117,3cyclictest0-21swapper/207:45:152
27169991110,1cyclictest0-21swapper/211:42:532
27169991110,1cyclictest0-21swapper/211:42:522
2716999110,10cyclictest4518-21cut07:30:142
2716799118,2cyclictest3520-21ssh12:31:541
2716799118,2cyclictest28179-21cut07:10:161
2716799118,2cyclictest1967-21ssh11:33:191
2716799118,2cyclictest18927-21cut10:45:171
2716799118,2cyclictest17785-21cron09:05:011
2716799118,2cyclictest14394-21grep08:55:211
2716799118,2cyclictest11121-21ssh11:38:511
2716799118,2cyclictest10562-21ssh11:57:381
2716799117,3cyclictest2924-21ssh10:55:151
2716799115,5cyclictest20916-21munin-node08:20:481
2716799111,9cyclictest25048-21diskmemload12:05:541
2716799110,10cyclictest22731-21processes08:05:231
2716399119,1cyclictest3680-21ssh11:15:140
2716399119,1cyclictest31412-21tail07:15:270
2716399119,1cyclictest27900-21grep08:15:270
2716399119,1cyclictest1148-21head09:55:280
2716399118,2cyclictest8259-21wc08:45:130
2716399118,2cyclictest5872-21rm11:55:160
2716399118,2cyclictest4092-21ssh12:32:210
2716399118,2cyclictest29295-21grep08:20:170
2716399118,2cyclictest29167-21cut09:15:150
2716399118,2cyclictest25438-21sed08:10:280
2716399118,2cyclictest20179-21tr08:00:210
2716399118,2cyclictest19955-21chrt09:07:270
2716399118,2cyclictest19223-21ssh10:25:280
2716399118,2cyclictest16916-21tr07:55:160
2716399118,2cyclictest11433-21ssh11:39:060
2716399111,4cyclictest0-21swapper/011:10:100
2716399111,4cyclictest0-21swapper/009:45:140
2716399110,8cyclictest398-21apt-get10:15:130
2716399110,2cyclictest452-21systemd-logind09:20:210
2716399110,2cyclictest20178-21diskmemload09:30:070
2716399110,2cyclictest14470-21/usr/sbin/munin07:50:140
2717299108,1cyclictest9624-21ls07:40:163
2717299108,1cyclictest292282sleep307:11:383
2717299108,1cyclictest2700-21df07:25:173
2717299108,1cyclictest199422sleep309:07:183
2717299108,1cyclictest19038-21basename08:00:113
2717299108,1cyclictest1173-21ssh10:35:033
2717299107,2cyclictest31448-21df_abs08:25:173
2716999108,2cyclictest31305-21expr08:25:172
2716999108,1cyclictest29880-21tr07:15:152
2716999100,9cyclictest18062-21seq09:05:142
2716999100,9cyclictest17809-21cat07:55:232
2716799108,1cyclictest9122sleep108:30:141
2716799108,1cyclictest9122sleep108:30:131
2716799108,1cyclictest11376-21chrt07:44:261
2716799107,2cyclictest7493-21chrt08:41:191
2716799107,2cyclictest4939-21sed08:35:271
2716799107,2cyclictest31229-21cat07:15:261
2716799107,2cyclictest24429-21tune2fs08:10:181
2716799106,3cyclictest11659-21wc08:50:201
2716799100,9cyclictest31091-21cut08:25:161
2716399108,1cyclictest77202sleep008:44:140
2716399108,1cyclictest5237-21grep07:30:180
2716399108,1cyclictest3934-21taskset07:26:190
2716399108,1cyclictest31268-21cat08:25:170
2716399108,1cyclictest27886-21sed07:10:150
2716399108,1cyclictest22254-21idleruntime08:05:170
2716399108,1cyclictest1440-21bc10:35:130
2716399107,2cyclictest6902-21cut07:35:140
2716399107,2cyclictest32542-21cat07:20:150
2716399107,2cyclictest13755-21chrt07:46:450
2716399107,2cyclictest10759-21sed07:40:270
271729997,1cyclictest23945-21wget08:10:163
271699997,1cyclictest26835-21uniq08:15:162
271679997,1cyclictest19174-21cut08:00:151
271639996,2cyclictest17803-21/usr/sbin/munin09:05:010
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional