You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-22 - 12:56

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Fri May 22, 2026 00:43:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1796929783,9sleep00-21swapper/019:09:140
1785728875,9sleep20-21swapper/219:07:482
1787128568,13sleep10-21swapper/119:07:591
1773328370,9sleep30-21swapper/319:06:123
80562640,1sleep30-21swapper/300:01:323
1811599270,27cyclictest0-21swapper/222:24:172
1811599260,25cyclictest0-21swapper/200:00:122
18108992623,2cyclictest13-21ksoftirqd/023:05:180
1810899260,1cyclictest0-21swapper/021:32:070
18115992523,2cyclictest0-21swapper/222:57:032
18115992521,3cyclictest0-21swapper/200:25:232
1811599250,24cyclictest0-21swapper/223:25:082
18118992421,2cyclictest1071-21kworker/3:223:41:233
18118992420,3cyclictest16259-21kworker/3:100:11:533
18115992418,5cyclictest0-21swapper/200:10:032
1811599240,23cyclictest0-21swapper/223:15:382
1811299241,21cyclictest20712-21open_inodes20:20:201
1811299240,2cyclictest0-21swapper/122:59:361
1811899230,22cyclictest1071-21kworker/3:223:16:133
1811599230,21cyclictest0-21swapper/222:11:482
1810899235,2cyclictest141rcu_preempt22:45:210
18118992219,2cyclictest1071-21kworker/3:222:49:533
1811899221,20cyclictest1071-21kworker/3:222:29:133
1811899220,22cyclictest1071-21kworker/3:223:12:133
1811299220,22cyclictest0-21swapper/122:07:441
1811899212,4cyclictest1071-21kworker/3:223:31:173
1811599210,21cyclictest0-21swapper/222:16:222
1811299210,1cyclictest0-21swapper/123:46:581
1810899212,17cyclictest2316-1kworker/u9:121:55:060
18108992121,0cyclictest0-21swapper/022:54:000
18118992018,2cyclictest16259-21kworker/3:100:25:333
18118992018,2cyclictest1071-21kworker/3:223:47:533
1811899200,19cyclictest1071-21kworker/3:222:57:133
18115992020,0cyclictest0-21swapper/222:42:592
1811599200,20cyclictest0-21swapper/222:06:092
1811599200,1cyclictest10705-21diskmemload22:50:232
1811599200,19cyclictest0-21swapper/223:56:122
1811599200,19cyclictest0-21swapper/221:50:072
1811599200,17cyclictest0-21swapper/200:05:122
18108992019,1cyclictest13-21ksoftirqd/000:06:570
18108992019,1cyclictest0-21swapper/021:52:580
18108992015,4cyclictest13-21ksoftirqd/022:28:170
1810899200,3cyclictest13-21ksoftirqd/023:45:420
18118991917,2cyclictest16259-21kworker/3:100:16:433
18118991917,2cyclictest1071-21kworker/3:223:56:533
1811899190,16cyclictest0-21swapper/323:35:413
1811599190,4cyclictest0-21swapper/221:10:382
1811299190,1cyclictest0-21swapper/122:41:291
1810899195,14cyclictest13-21ksoftirqd/000:00:190
18108991911,6cyclictest13-21ksoftirqd/022:04:580
18108991911,4cyclictest13-21ksoftirqd/023:41:310
18108991910,9cyclictest13-21ksoftirqd/021:37:460
18118991816,2cyclictest1071-21kworker/3:223:51:333
18118991816,2cyclictest1071-21kworker/3:222:15:233
1811599182,1cyclictest10705-21diskmemload23:31:172
1811599180,2cyclictest0-21swapper/222:27:532
1811299182,13cyclictest10705-21diskmemload21:23:171
18112991818,0cyclictest0-21swapper/123:05:341
18112991816,1cyclictest0-21swapper/122:01:281
1811299181,1cyclictest10705-21diskmemload23:36:251
1810899183,6cyclictest3531-21ssh22:22:170
1810899183,1cyclictest141rcu_preempt23:04:110
1810899180,15cyclictest0-21swapper/023:10:510
1811899174,5cyclictest225-21systemd-journal20:20:003
18118991715,2cyclictest16259-21kworker/3:100:24:533
18118991715,2cyclictest16259-21kworker/3:100:24:533
18115991715,2cyclictest0-21swapper/223:46:032
18115991713,3cyclictest31203-21ssh23:36:432
18115991713,3cyclictest0-21swapper/200:24:432
18115991713,3cyclictest0-21swapper/200:24:422
18112991714,2cyclictest10705-21diskmemload22:24:201
1811299170,17cyclictest0-21swapper/100:21:081
1811299170,17cyclictest0-21swapper/100:21:081
1811299170,17cyclictest0-21swapper/100:01:311
1810899174,5cyclictest1142-21runrttasks00:26:500
1811899163,1cyclictest18051-21ssh21:14:033
18118991614,2cyclictest1071-21kworker/3:223:25:333
18118991614,2cyclictest1071-21kworker/3:223:23:233
18118991614,2cyclictest1071-21kworker/3:222:09:033
1811599162,14cyclictest56550irq/124-enp0s3119:18:132
1811599160,16cyclictest0-21swapper/221:56:292
1811599160,15cyclictest0-21swapper/223:10:242
1811599160,15cyclictest0-21swapper/200:36:112
18112991615,1cyclictest0-21swapper/119:49:181
18112991614,2cyclictest0-21swapper/121:30:121
1811299160,15cyclictest0-21swapper/123:30:261
1810899168,8cyclictest13-21ksoftirqd/000:36:440
1810899166,7cyclictest537-21snmpd21:48:030
1810899163,7cyclictest32438-21awk19:40:000
1810899162,7cyclictest30653-21munin-run00:35:000
1810899162,3cyclictest13-21ksoftirqd/021:40:170
18108991614,2cyclictest161rcuc/021:15:500
18108991614,2cyclictest161rcuc/020:25:210
1810899160,3cyclictest0-21swapper/022:36:300
1810899160,3cyclictest0-21swapper/022:36:300
1810899160,15cyclictest0-21swapper/000:18:310
18118991512,2cyclictest0-21swapper/321:59:543
18112991515,0cyclictest0-21swapper/119:40:181
18108991511,1cyclictest0-21swapper/023:38:300
18118991411,2cyclictest20299-21head20:20:153
18118991410,3cyclictest9861-21grep20:00:133
18118991410,3cyclictest8273-21cat22:45:013
18118991410,3cyclictest31313-21cut00:35:143
18115991411,3cyclictest0-21swapper/222:49:032
18115991410,3cyclictest18269-21tr20:15:202
1811599140,1cyclictest0-21swapper/221:39:242
1811299142,11cyclictest0-21swapper/122:28:401
18112991410,3cyclictest24582-21tr22:15:221
1811299140,14cyclictest0-21swapper/121:14:011
1811299140,0cyclictest0-21swapper/100:27:381
1810899147,7cyclictest13-21ksoftirqd/022:12:200
1810899145,1cyclictest141rcu_preempt23:27:360
18108991411,2cyclictest13-21ksoftirqd/020:15:000
18108991410,3cyclictest13-21ksoftirqd/023:20:090
18108991410,2cyclictest23355-21grep19:20:130
1810899140,1cyclictest0-21swapper/000:23:090
1810899140,1cyclictest0-21swapper/000:23:090
1811899138,2cyclictest14606-21ssh21:50:193
18118991312,1cyclictest10705-21diskmemload00:31:043
18118991310,2cyclictest9500-21sh20:00:003
18118991310,2cyclictest30945-21grep19:35:163
18118991310,2cyclictest25989-21grep19:25:153
18118991310,2cyclictest23815-21sort19:20:303
18118991310,2cyclictest18651-21ls22:50:253
18118991310,2cyclictest13125-21cat20:05:173
18115991310,3cyclictest0-21swapper/223:50:332
18115991310,2cyclictest4820-21ntpq23:40:192
18115991310,2cyclictest32271-21sh22:40:002
18115991310,2cyclictest32271-21sh22:40:002
18115991310,2cyclictest31904-21wc00:15:262
18115991310,2cyclictest31723-21grep20:45:132
18115991310,2cyclictest28419-21grep19:30:152
1811299139,3cyclictest5475-21tr21:45:141
1811299139,3cyclictest31814-21sort19:35:241
1811299136,6cyclictest0-21swapper/122:45:241
18112991311,1cyclictest0-21swapper/122:34:011
1811299131,10cyclictest10705-21diskmemload23:12:501
18112991310,2cyclictest6918-21grep21:00:171
18112991310,2cyclictest23186-21grep21:55:221
18112991310,2cyclictest1870-21sleep23:20:001
18112991310,2cyclictest17546-21cat00:07:061
18112991310,2cyclictest10561-21sed20:00:161
1811299130,1cyclictest0-21swapper/123:53:191
1811299130,11cyclictest16728-21tail22:10:261
1810899139,3cyclictest30713-21grep20:40:250
1810899139,3cyclictest25469-21wget22:55:150
1810899139,3cyclictest23480-21ssh00:10:260
1810899139,3cyclictest19054-21grep19:10:170
1810899139,2cyclictest5370-21grep21:25:190
1810899133,5cyclictest9521-21sh20:00:000
18108991312,1cyclictest161rcuc/023:55:060
18108991311,2cyclictest13-21ksoftirqd/022:20:000
18108991311,2cyclictest0-21swapper/022:05:120
18108991310,2cyclictest12688-21tr21:10:200
18108991310,2cyclictest0-21swapper/021:21:250
1810899130,2cyclictest0-21swapper/023:15:030
1810899130,12cyclictest0-21swapper/023:50:310
1811899129,2cyclictest5896-21uniq21:45:163
1811899129,2cyclictest3586-21ssh22:03:093
1811899129,2cyclictest32207-21grep20:45:153
1811899129,2cyclictest28375-21sed20:35:253
1811899129,2cyclictest28156-21ssh21:20:133
1811899129,2cyclictest24330-21tr22:35:143
1811899129,2cyclictest24330-21tr22:35:143
1811899129,2cyclictest23520-21grep20:25:263
1811899129,2cyclictest20412-21ssh21:35:003
1811899129,2cyclictest19081-21head19:10:173
1811899129,2cyclictest15131-21grep00:05:273
1811899129,2cyclictest14991-21cut22:10:143
1811899129,2cyclictest1023-21cut19:40:153
1811899126,4cyclictest11958-21cron21:30:003
1811899123,5cyclictest225-21systemd-journal20:15:003
1811599129,2cyclictest5623-21cut19:50:142
1811599129,2cyclictest4297-21grep23:20:262
1811599129,2cyclictest3249-21head23:00:262
1811599129,2cyclictest26731-21tr19:25:222
1811599129,2cyclictest25230-21sh19:25:002
1811599129,2cyclictest23739-21sh22:35:002
1811599129,2cyclictest23113-21grep00:30:162
1811599129,2cyclictest21213-21cut21:15:232
1811599129,2cyclictest12436-21tr20:05:132
1811599129,2cyclictest11390-21sort23:05:262
1811599129,2cyclictest10698-21turbostat21:10:002
1811599129,1cyclictest10705-21diskmemload21:31:052
1811599127,2cyclictest225-21systemd-journal20:55:002
1811599127,2cyclictest225-21systemd-journal20:55:002
1811599121,3cyclictest28849-21kworker/u8:022:02:422
1811599121,2cyclictest6446-21grep21:45:192
18115991212,0cyclictest0-21swapper/221:42:592
1811599121,10cyclictest56550irq/124-enp0s3119:35:152
1811299129,2cyclictest9224-21tune2fs21:05:181
1811299129,2cyclictest6127-21wc21:00:011
1811299129,2cyclictest28642-21tail19:30:161
1811299129,2cyclictest26586-21tail19:25:211
1811299129,2cyclictest24829-21grep00:30:291
1811299129,2cyclictest19075-21head19:10:171
1811299129,2cyclictest18196-21tr22:50:201
1811299129,2cyclictest1727-21tr20:50:131
1811299129,2cyclictest1727-21tr20:50:131
1811299127,4cyclictest27794-21ssh00:13:341
1811299127,4cyclictest25799-21sed20:30:251
1811299120,3cyclictest0-21swapper/121:35:171
1810899129,2cyclictest5464-21tail20:55:250
1810899129,2cyclictest20523-21ls23:30:220
1810899129,2cyclictest1689-21grep20:50:130
1810899129,2cyclictest1689-21grep20:50:130
1810899129,2cyclictest10718-21http21:10:000
1810899128,3cyclictest28071-21cat20:35:240
1811899118,2cyclictest752-21ssh22:20:243
1811899118,2cyclictest31996-21ssh21:41:003
1811899118,2cyclictest29408-21wc20:40:133
1811899118,2cyclictest22212-21ssh21:15:453
1811899118,2cyclictest20856-21sh21:35:133
1811899118,2cyclictest19598-21ssh22:31:463
1811899118,2cyclictest1800-21expr23:00:153
1811899118,2cyclictest10052-21cat21:05:263
1811599118,2cyclictest9536-21tr20:00:002
1811599118,2cyclictest4949-21grep21:25:172
1811599118,2cyclictest30183-21grep21:20:272
1811599118,2cyclictest29823-21grep20:40:152
1811599118,2cyclictest27425-21grep20:35:152
1811599118,2cyclictest22011-21grep20:25:132
1811599118,2cyclictest19131-21sed19:10:172
1811599113,7cyclictest56550irq/124-enp0s3121:00:212
1811299119,1cyclictest0-21swapper/121:25:591
1811299118,2cyclictest4281-21grep23:40:161
1811299118,2cyclictest32418-21ssh21:41:201
1811299118,2cyclictest31200-21cut00:35:131
1811299118,2cyclictest30298-21ssh23:55:241
1811299118,2cyclictest28303-21sed20:35:241
1811299118,2cyclictest25014-21taskset19:22:201
1811299118,2cyclictest24176-21chrt22:35:101
1811299118,2cyclictest24176-21chrt22:35:101
1811299118,2cyclictest23838-21chrt20:27:521
1811299118,2cyclictest2096-21grep23:00:171
1811299118,2cyclictest19585-21ssh21:53:341
1811299118,2cyclictest18831-21head20:15:261
1811299118,2cyclictest16694-21chrt20:12:571
1811299118,2cyclictest12144-21ssh23:25:221
1811299110,10cyclictest5996-21ssh23:21:421
1811299110,10cyclictest24648-21ssh21:17:401
1810899118,2cyclictest6231-21cut21:00:140
1810899118,2cyclictest26539-21wc20:35:000
1810899118,2cyclictest2363-21taskset19:44:520
1810899118,2cyclictest21483-21head19:15:160
1810899118,2cyclictest16265-21ssh22:30:150
1810899112,7cyclictest13-21ksoftirqd/022:42:380
1811899108,1cyclictest9545-21ssh23:05:103
1811899108,1cyclictest6838-21taskset19:51:213
1811899108,1cyclictest44042sleep319:45:353
1811899108,1cyclictest21300-21df_inode19:15:153
1811899108,1cyclictest1407-21basename20:50:103
1811899108,1cyclictest1407-21basename20:50:103
1811899107,2cyclictest7984-21chrt21:00:563
1811899107,2cyclictest27838-21sshd19:30:103
1811899107,2cyclictest24238-21taskset20:30:033
1811899105,2cyclictest225-21systemd-journal20:55:083
1811599108,1cyclictest2025-21taskset19:40:302
1811599107,2cyclictest3771-21chrt20:55:062
1811599107,2cyclictest2911-21tr19:45:142
1811599107,2cyclictest24245-21chrt20:30:092
1811599107,2cyclictest16408-21grep20:10:252
1811599100,9cyclictest10413-21taskset20:00:152
1811299108,1cyclictest91712sleep119:55:501
1811299108,1cyclictest808-21seq20:45:381
1811299108,1cyclictest5138-21cut19:50:121
1811299108,1cyclictest32274-21ssh00:15:321
1811299107,2cyclictest30478-21tr20:40:211
1811299107,2cyclictest21440-21head19:15:161
1811299107,2cyclictest12296-21cut20:05:121
1810899108,1cyclictest3476-21ls19:45:170
1810899108,1cyclictest29103-21open_files19:30:210
1810899108,1cyclictest21578-21chrt20:24:180
1810899107,2cyclictest18194-21cat20:15:200
1810899107,2cyclictest12879-21grep20:05:160
1810899105,2cyclictest161rcuc/019:25:170
1810899104,2cyclictest452-21systemd-logind20:02:560
181159996,2cyclictest19543-21aten2_r7power_c20:20:122
181089997,1cyclictest32242-21expr20:45:160
181089980,5cyclictest13-21ksoftirqd/019:50:130
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional