You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-10 - 09:59

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot7s.osadl.org (updated Sun May 10, 2026 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
265852910,0sleep00-21swapper/023:05:110
117829066,8sleep00-21swapper/019:08:150
109728864,19sleep20-21swapper/219:07:172
121928565,16sleep30-21swapper/319:08:453
126428366,13sleep10-21swapper/119:09:201
321832760,0sleep10-21swapper/122:49:061
179862290,0chrt17987-21sh23:57:323
1416992927,2cyclictest39-21ksoftirqd/323:48:473
141499280,27cyclictest0-21swapper/222:40:392
141499280,27cyclictest0-21swapper/222:40:382
1408992825,2cyclictest13-21ksoftirqd/021:10:000
140899273,3cyclictest141rcu_preempt21:10:260
1408992720,6cyclictest13-21ksoftirqd/021:50:190
141499260,22cyclictest0-21swapper/221:21:302
141499250,25cyclictest0-21swapper/221:36:012
140899253,22cyclictest13-21ksoftirqd/021:18:170
1408992321,2cyclictest13-21ksoftirqd/000:36:330
141699220,0cyclictest0-21swapper/322:09:333
1408992221,1cyclictest13-21ksoftirqd/021:46:480
141499210,21cyclictest0-21swapper/223:05:552
141499210,21cyclictest0-21swapper/222:13:432
141199217,1cyclictest20503-21apt-get21:25:121
140899214,11cyclictest13-21ksoftirqd/000:19:400
140899212,5cyclictest13-21ksoftirqd/000:27:390
140899212,5cyclictest13-21ksoftirqd/000:27:380
140899212,19cyclictest13-21ksoftirqd/023:15:510
1408992120,1cyclictest13-21ksoftirqd/023:12:020
1408992119,2cyclictest13-21ksoftirqd/023:03:260
141699201,0cyclictest0-21swapper/322:31:353
141699200,2cyclictest0-21swapper/321:34:393
140899207,2cyclictest141rcu_preempt22:21:480
140899204,8cyclictest13-21ksoftirqd/022:19:240
140899202,18cyclictest0-21swapper/020:45:140
1408992018,1cyclictest31001-21apt-get22:10:130
141699190,19cyclictest0-21swapper/323:07:303
141699190,18cyclictest0-21swapper/322:45:163
141499191,4cyclictest141rcu_preempt19:45:252
1414991914,1cyclictest32-21ksoftirqd/222:15:222
141499190,19cyclictest0-21swapper/222:54:562
141199192,16cyclictest27047-21diskmemload00:08:351
141199192,15cyclictest27047-21diskmemload23:11:321
1411991918,1cyclictest0-21swapper/119:34:541
141199190,18cyclictest0-21swapper/123:16:051
141199190,14cyclictest0-21swapper/121:39:441
1408991918,1cyclictest13-21ksoftirqd/023:38:580
1408991916,2cyclictest13-21ksoftirqd/023:22:260
1408991913,5cyclictest141rcu_preempt22:54:140
1414991816,1cyclictest31249-21ssh00:25:052
1414991816,1cyclictest31249-21ssh00:25:042
141499180,4cyclictest0-21swapper/223:53:482
141499180,17cyclictest0-21swapper/220:40:132
141199185,8cyclictest0-21swapper/122:23:501
140899186,6cyclictest32014-21awk22:30:010
140899180,1cyclictest0-21swapper/019:30:130
141499172,8cyclictest29333-21sensors23:25:252
1411991717,0cyclictest0-21swapper/100:31:091
141199170,1cyclictest0-21swapper/123:34:351
141199170,14cyclictest0-21swapper/121:23:501
141199170,0cyclictest0-21swapper/100:28:241
141199170,0cyclictest0-21swapper/100:28:231
140899172,9cyclictest3005-21grep20:19:590
140899172,5cyclictest21194-21basename23:59:590
1408991711,5cyclictest13-21ksoftirqd/023:51:230
140899170,1cyclictest0-21swapper/000:00:400
1416991615,1cyclictest0-21swapper/321:23:173
1416991614,1cyclictest0-21swapper/322:10:123
141699160,2cyclictest0-21swapper/323:19:153
1414991615,1cyclictest0-21swapper/223:32:452
1414991611,3cyclictest28535-21ssh21:30:002
141499160,2cyclictest0-21swapper/200:31:362
141499160,1cyclictest0-21swapper/223:49:332
141499160,1cyclictest0-21swapper/222:23:542
141499160,16cyclictest0-21swapper/200:03:342
140899162,8cyclictest13851-21idleruntime-cro21:59:590
1408991616,0cyclictest0-21swapper/019:55:580
140899160,3cyclictest0-21swapper/022:38:460
140899160,3cyclictest0-21swapper/000:23:150
1416991515,0cyclictest0-21swapper/300:26:563
1416991515,0cyclictest0-21swapper/300:26:553
141699150,1cyclictest0-21swapper/322:37:343
141499154,3cyclictest12642-21apache222:09:252
1414991512,2cyclictest0-21swapper/222:35:122
141499150,0cyclictest0-21swapper/200:38:402
141499150,0cyclictest0-21swapper/200:06:112
141199159,6cyclictest0-21swapper/122:02:101
141199150,15cyclictest0-21swapper/123:27:251
141199150,15cyclictest0-21swapper/122:59:341
140899159,4cyclictest27047-21diskmemload23:25:370
140899158,3cyclictest161rcuc/021:35:160
140899157,7cyclictest24679-21munin-run21:05:010
140899155,2cyclictest141rcu_preempt22:05:250
140899153,11cyclictest0-21swapper/021:30:500
140899152,2cyclictest0-21swapper/020:35:130
1408991512,3cyclictest12638-21apache223:47:240
140899150,1cyclictest0-21swapper/021:41:570
140899150,1cyclictest0-21swapper/019:50:120
141699149,3cyclictest14925-21cron20:45:013
1416991413,1cyclictest0-21swapper/323:24:563
1416991410,2cyclictest17411-21cut22:40:143
1416991410,2cyclictest17411-21cut22:40:133
141499143,10cyclictest22362-21/usr/sbin/munin23:40:232
141499140,9cyclictest0-21swapper/222:25:132
141499140,1cyclictest0-21swapper/223:23:462
141499140,14cyclictest0-21swapper/223:01:382
141499140,0cyclictest0-21swapper/222:56:392
141199149,4cyclictest20425-21grep19:45:251
1411991411,2cyclictest6999-21wc21:55:191
1411991410,3cyclictest11102-21cron19:30:011
1411991410,2cyclictest26103-21apt23:05:011
141199140,1cyclictest0-21swapper/123:45:441
141199140,13cyclictest25659-21munin-run20:00:001
140899147,3cyclictest225-21systemd-journal20:25:000
140899146,6cyclictest11455-21kworker/0:122:32:120
140899143,2cyclictest0-21swapper/020:55:140
140899141,6cyclictest18533-21munin-run19:45:000
140899141,2cyclictest141rcu_preempt00:13:220
140899141,1cyclictest0-21swapper/021:21:150
1408991411,3cyclictest13-21ksoftirqd/000:30:540
141699138,3cyclictest5119-21tr23:50:133
141699131,3cyclictest13805-21sh22:56:573
1416991310,2cyclictest8765-21tr00:30:213
1416991310,2cyclictest8133-21sed00:10:253
1416991310,2cyclictest6739-21sed21:35:263
1416991310,2cyclictest29924-21grep20:05:253
1416991310,2cyclictest28561-21tail23:25:173
1416991310,2cyclictest27841-21basename21:10:153
1416991310,2cyclictest24094-21cat21:00:263
1416991310,2cyclictest21879-21sed20:55:273
1416991310,2cyclictest17016-21sort19:40:173
1416991310,2cyclictest16683-21head00:35:163
1416991310,2cyclictest16532-21ssh00:15:253
141699130,1cyclictest0-21swapper/321:25:193
141699130,13cyclictest0-21swapper/321:40:513
141699130,12cyclictest1565-21ssh22:50:113
141499139,3cyclictest987-21cat22:50:012
141499133,5cyclictest10412-21cron23:15:012
1414991312,1cyclictest31912-21memory20:10:182
1414991310,2cyclictest5905-21tr20:25:142
1414991310,2cyclictest18810-21cron23:19:592
1414991310,2cyclictest18171-21grep20:50:162
1414991310,2cyclictest17442-21cat19:40:202
1414991310,2cyclictest15017-21cat23:55:252
141499130,2cyclictest0-21swapper/221:32:272
141499130,10cyclictest0-21swapper/220:30:142
141499130,0cyclictest0-21swapper/221:41:092
141199139,3cyclictest5607-21sh21:15:261
141199135,5cyclictest29154-21ssh00:23:371
141199131,1cyclictest141rcu_preempt23:22:381
1411991310,2cyclictest19064-21grep22:40:251
1411991310,2cyclictest19064-21grep22:40:241
141199130,1cyclictest0-21swapper/122:34:141
141199130,1cyclictest0-21swapper/121:48:001
141199130,11cyclictest0-21swapper/121:12:361
140899139,2cyclictest8035-21grep20:30:140
140899138,2cyclictest13-21ksoftirqd/019:20:010
140899134,2cyclictest13-21ksoftirqd/023:31:450
1408991312,1cyclictest13-21ksoftirqd/022:40:110
1408991312,1cyclictest13-21ksoftirqd/022:40:100
140899130,3cyclictest0-21swapper/022:00:130
141699129,2cyclictest8707-21grep20:30:183
141699129,2cyclictest4505-21tr19:15:163
141699129,2cyclictest3236-21sort23:10:163
141699129,2cyclictest32274-21cut20:10:213
141699129,2cyclictest21942-21grep19:50:173
141699129,2cyclictest21922-21tr21:45:163
141699129,2cyclictest15742-21cut22:20:143
141699129,2cyclictest14974-21sed22:00:163
141699128,3cyclictest2531-21sed20:15:273
141699128,3cyclictest15055-21sh22:19:593
141699124,4cyclictest12053-21apt-get23:35:123
141699122,1cyclictest31227-21ntp_states21:50:183
141499129,2cyclictest7821-21ls19:20:212
141499129,2cyclictest5490-21wc19:15:232
141499129,2cyclictest31755-21cat21:50:242
141499129,2cyclictest24002-21sort21:00:262
141499129,2cyclictest2029-21tail22:30:252
141499129,2cyclictest11360-21wc20:35:182
141499120,1cyclictest0-21swapper/221:48:002
141499120,1cyclictest0-21swapper/200:12:252
141499120,11cyclictest29361-21cat20:05:192
141199129,2cyclictest8346-21head19:20:251
141199129,2cyclictest7885-21sort22:15:171
141199129,2cyclictest4964-21grep20:20:261
141199129,2cyclictest27568-21chrt22:26:341
141199129,2cyclictest21937-21cut19:50:171
141199129,2cyclictest14432-21wc21:40:201
141199128,3cyclictest17760-21grep00:35:261
141199128,3cyclictest16827-21sed20:45:271
141199128,1cyclictest25-21ksoftirqd/123:54:311
141199127,4cyclictest1370-21apache220:25:111
141199127,3cyclictest30849-21apt-get20:10:021
141199126,5cyclictest537-21snmpd00:03:171
141199121,3cyclictest225-21systemd-journal22:38:011
1411991211,1cyclictest0-21swapper/100:16:211
141199120,1cyclictest27047-21diskmemload21:32:461
141199120,12cyclictest0-21swapper/121:54:101
141199120,11cyclictest6372-21basename19:20:011
141199120,0cyclictest0-21swapper/123:56:091
140899129,2cyclictest6398-21cut20:25:160
140899129,2cyclictest31688-21sensors00:05:220
140899129,2cyclictest27439-21grep22:45:280
140899129,2cyclictest13-21ksoftirqd/019:35:140
140899125,6cyclictest573-21apt20:15:000
141699118,2cyclictest6908-21ssh21:16:203
141699118,2cyclictest6383-21ls21:55:153
141699118,2cyclictest4961-21sh23:30:213
141699118,2cyclictest2811-21taskset00:07:423
141699118,2cyclictest25685-21grep21:05:183
141699118,2cyclictest25362-21ssh00:01:433
141699118,2cyclictest25328-21cat22:25:193
141699118,2cyclictest24266-21head19:55:173
141699118,2cyclictest23141-21tr00:20:153
141699118,2cyclictest19049-21expr19:45:143
141699118,2cyclictest18673-21cat23:00:133
141699118,2cyclictest11669-21cstates19:30:133
141699118,2cyclictest11605-21head20:35:213
141699118,2cyclictest10884-21seq19:27:183
141699117,3cyclictest6295-21grep20:25:163
141699117,3cyclictest21290-21sed23:40:153
141499119,1cyclictest5153-21grep21:15:222
141499119,1cyclictest24936-21apt-get21:05:132
141499119,1cyclictest15966-21head20:45:182
141499118,2cyclictest2616-21grep19:10:292
141499118,2cyclictest16056-21head00:15:212
141499118,2cyclictest15638-21grep19:35:252
141499118,2cyclictest14497-21hostname22:00:152
141499118,2cyclictest1381-21ssh21:13:352
141499118,2cyclictest13430-21ntpq23:35:212
141499118,2cyclictest11620-21grep19:30:132
141499118,2cyclictest10754-21taskset19:25:352
141499118,2cyclictest10305-21taskset21:57:182
141499115,4cyclictest23025-21/usr/sbin/munin00:20:292
141499110,9cyclictest0-21swapper/220:20:132
141199119,1cyclictest20065-21cat23:40:001
141199118,2cyclictest9087-21ssh22:54:491
141199118,2cyclictest27403-21grep23:05:161
141199118,2cyclictest23777-21grep21:00:231
141199118,2cyclictest2341-21wc20:15:251
141199118,2cyclictest20873-21grep20:55:181
141199118,2cyclictest17709-21wget20:50:131
141199118,2cyclictest15922-21chrt19:37:271
141199115,5cyclictest23425-21sort22:05:161
141199115,5cyclictest14548-21head20:40:261
141199110,10cyclictest7708-21ssh00:10:211
140899118,2cyclictest9225-21tr19:25:140
140899118,2cyclictest3377-21wc19:10:370
140899118,2cyclictest30433-21awk20:10:000
140899118,2cyclictest25798-21ssh21:27:590
140899118,2cyclictest12883-21ssh22:56:130
140899117,1cyclictest26638-21expr20:00:150
140899116,4cyclictest161rcuc/023:40:180
140899111,6cyclictest13-21ksoftirqd/020:55:010
141699108,1cyclictest6633-21apt-get19:20:123
141699108,1cyclictest17715-21cpu20:50:143
141699108,1cyclictest17188-21taskset20:48:363
141699108,1cyclictest13976-21chrt19:35:023
141699107,2cyclictest3367-21cut20:20:143
141699106,3cyclictest26210-21grep20:00:143
141499108,1cyclictest26235-21cat20:00:132
141499108,1cyclictest25570-21chrt19:59:132
141499108,1cyclictest21382-21grep19:50:142
141499108,1cyclictest1782-21uname20:15:202
141499107,2cyclictest20097-21sed20:55:012
141199108,1cyclictest23182-21ssh23:40:571
141199108,1cyclictest16620-21grep19:40:151
141199107,2cyclictest8327-21expr20:30:151
141199107,2cyclictest31721-21tune2fs22:10:171
141199107,2cyclictest30337-21chrt20:09:101
141199107,2cyclictest25983-21grep20:00:121
141199107,2cyclictest2536-21grep19:10:291
141199106,3cyclictest11110-21tail20:35:171
140899108,1cyclictest20579-21chrt19:46:050
140899108,1cyclictest0-21swapper/019:20:130
14169997,1cyclictest18202sleep319:10:143
14119997,1cyclictest25473-21df_abs21:05:151
14089993,2cyclictest13-21ksoftirqd/020:40:170
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional